Why Your Inference Bill Doesn't Reflect Your Usage

The invoice says you spent $34,000 on LLM inference last month. It doesn’t say who, what, or why. A practical guide to building the attribution system that turns a billing line item into an operational signal.

This pairs with Layer 9 of the twelve layers of LLM cost, the API spend flywheel, model escalation, and the audit trail you can’t reconstruct — attribution tells you where spend went; budgets, tiering, and audit trails tell you what to do about it.

The bill that surprised everyone

In February 2026, an operations platform opened its LLM provider invoice and saw $34,000 in inference spend.

January had been $9,000.

The product team had expected an increase. They had launched a document summarization feature for customer workspaces. The feature took uploaded PDFs, summarized the contents, extracted action items, and wrote a short explanation back into the workspace activity feed.

What nobody expected was the scale: the summarization feature had made 60,000 calls in February. Some calls summarized short memos. Others summarized 200-page packets. Some workspaces triggered summarization every time a file changed. A few agent loops retried failed summaries with the full document context again and again.

The provider dashboard showed spend by model and day. It did not answer the questions leadership asked:

Which team owned the 60,000 calls?
Which customer workspaces drove the spike?
Which feature path was responsible?
How much came from routine summarization versus retries?
Why did a few calls cost several dollars each?
Would March be $34,000 again, or was February an anomaly?

The team spent a week joining provider exports, application logs, and deployment timelines. They eventually found the culprit: a three-day spike after document summarization launched with no per-team virtual key, no feature-level attribution, and no budget gate. The invoice was accurate. It just wasn’t useful.

Lesson: the provider invoice is a receipt. Attribution is the system that turns the receipt into an operational signal.

Why provider dashboards fail

Provider dashboards answer provider questions: how many tokens were billed, which model was used, and what the invoice total is.

Production teams need different dimensions:

Question	Provider dashboard	Attribution system
Who spent the money?	Sometimes API key	Team, service, workspace, virtual key
What feature spent it?	No	Operation and feature tags
Why did it happen?	No	Correlation ID, trace, retry reason, model escalation
Was it valuable?	No	Verdict and downstream outcome
Could it have been prevented?	No	Budget gate, rate limit, policy
What should change?	No	Tier distribution and optimization loop

API keys are too coarse. A shared key used by three services creates one billing line. Models are too coarse. “$18,000 on Claude” does not tell you whether the spend came from summarization, support chat, extraction, or a runaway agent. Tokens are too coarse. A 50,000-token call might be legitimate legal synthesis or a loop that re-sent the same PDF five times.

Time aggregation is also misleading. February’s $34,000 total hid a three-day period where the new summarization workflow consumed most of its budget before anyone looked at the dashboard.

Lesson: attribution needs dimensions the provider cannot know: team, feature, operation, workspace, verdict, retry, budget, and tier.

The virtual key pattern

The first fix is to stop sharing raw provider keys across teams and features. Every caller should use a virtual key issued by the gateway.

clawql inference keys create \
  --team documents \
  --feature document-summarization \
  --budget 12000 \
  --period monthly \
  --default-operation summarize_document

clawql inference keys create \
  --team support \
  --feature support-bot \
  --budget 4000 \
  --period monthly \
  --default-operation answer_support_question

clawql inference keys create \
  --team experiments \
  --feature agent-lab \
  --budget 500 \
  --period monthly \
  --default-operation experimental_agent

Applications call the gateway with the virtual key:

from openai import OpenAI

client = OpenAI(
    base_url="https://inference.example.com/v1",
    api_key="vk_documents_2026_02",
)

response = client.chat.completions.create(
    model="clawql:auto",
    messages=[{"role": "user", "content": summarization_prompt}],
    extra_headers={
        "X-ClawQL-Workspace": "workspace_481",
        "X-ClawQL-Feature": "document-summarization",
        "X-ClawQL-Operation": "summarize_document",
        "X-ClawQL-Correlation-Id": correlation_id,
    },
)

The gateway maps virtual keys to defaults:

{
  "virtual_key_id": "vk_documents_2026_02",
  "team": "documents",
  "feature": "document-summarization",
  "default_operation": "summarize_document",
  "budget_usd": 12000,
  "period": "monthly",
  "allowed_environments": ["production"],
  "rate_limit_rpm": 600
}

If the application forgets to send a team header, the virtual key still supplies one. If it forgets feature or operation, defaults apply. Explicit request headers can refine attribution, but the call is never anonymous.

Lesson: attribution starts at the credential boundary. A raw provider key tells you the provider account. A virtual key tells you the operational owner.

BudgetGate

Attribution after the fact is useful. Prevention is better.

BudgetGate runs before the gateway forwards a request to the provider. It estimates the call cost, checks the virtual key’s remaining budget, reserves spend, and fails closed when the budget is exhausted.

class BudgetExhausted(Exception):
    def __init__(self, *, team, feature, budget_usd, current_spend_usd, estimated_cost_usd):
        self.team = team
        self.feature = feature
        self.budget_usd = budget_usd
        self.current_spend_usd = current_spend_usd
        self.estimated_cost_usd = estimated_cost_usd


class BudgetGate:
    def __init__(self, budget_store, price_table):
        self.budget_store = budget_store
        self.price_table = price_table

    def check_and_reserve(self, *, virtual_key, model, estimated_input_tokens, estimated_output_tokens):
        estimated_cost = self.price_table.estimate(
            model=model,
            input_tokens=estimated_input_tokens,
            output_tokens=estimated_output_tokens,
        )

        current_spend = self.budget_store.current_spend(
            virtual_key_id=virtual_key.id,
            period=virtual_key.period,
        )

        if current_spend + estimated_cost > virtual_key.budget_usd:
            raise BudgetExhausted(
                team=virtual_key.team,
                feature=virtual_key.feature,
                budget_usd=virtual_key.budget_usd,
                current_spend_usd=current_spend,
                estimated_cost_usd=estimated_cost,
            )

        self.budget_store.reserve(
            virtual_key_id=virtual_key.id,
            amount_usd=estimated_cost,
        )
        return estimated_cost

The caller receives a structured error:

{
  "error": "budget_exhausted",
  "team": "documents",
  "feature": "document-summarization",
  "budget_usd": 12000.0,
  "current_spend_usd": 11987.42,
  "estimated_cost_usd": 18.31,
  "period": "2026-02",
  "resets_at": "2026-03-01T00:00:00Z"
}

For the February spike, a $12,000 budget would have stopped the summarization feature before it reached $34,000. A 75% alert would have fired during the three-day spike. A per-workspace rate limit would have caught the loop even earlier.

Lesson: dashboards tell you spend happened. Budget gates decide whether spend is allowed to happen.

The call store SQL schema

The call store is the source of truth for attribution. It is append-only operational telemetry, not a monthly spreadsheet.

CREATE TABLE inference_calls (
  id UUID PRIMARY KEY,
  occurred_at TIMESTAMPTZ NOT NULL,
  correlation_id TEXT NOT NULL,
  request_id TEXT NOT NULL,

  team TEXT NOT NULL,
  feature TEXT NOT NULL,
  operation TEXT NOT NULL,
  workspace_id TEXT,
  virtual_key_id TEXT NOT NULL,
  environment TEXT NOT NULL,

  provider TEXT NOT NULL,
  model TEXT NOT NULL,
  tier TEXT NOT NULL CHECK (tier IN ('frugal', 'standard', 'frontier')),
  input_tokens INTEGER NOT NULL,
  output_tokens INTEGER NOT NULL,
  estimated_cost_usd NUMERIC(12, 6) NOT NULL,
  actual_cost_usd NUMERIC(12, 6),
  latency_ms INTEGER NOT NULL,

  cache_hit BOOLEAN NOT NULL DEFAULT false,
  retry_count INTEGER NOT NULL DEFAULT 0,
  escalation_from TEXT,
  escalation_reason TEXT,

  verdict TEXT CHECK (verdict IN ('passed', 'failed', 'hitl')),
  prompt_hash TEXT NOT NULL,
  response_hash TEXT,
  policy_manifest_hash TEXT,

  created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

CREATE INDEX inference_calls_time_idx
  ON inference_calls (occurred_at);

CREATE INDEX inference_calls_team_period_idx
  ON inference_calls (team, occurred_at);

CREATE INDEX inference_calls_feature_period_idx
  ON inference_calls (feature, occurred_at);

CREATE INDEX inference_calls_correlation_idx
  ON inference_calls (correlation_id);

The fields that matter most are team, feature, operation, workspace_id, virtual_key_id, tier, estimated_cost_usd, retry_count, and correlation_id.

The provider invoice can be reconciled later with actual_cost_usd. The gateway needs estimated_cost_usd immediately so budgets and alerts can work before the invoice arrives.

Lesson: if the call store lacks attribution dimensions, the finance team will recreate them later with guesswork. Capture them at the gateway instead.

Spend report SQL and CLI

Once calls are tagged, the February invoice becomes queryable.

SELECT
  feature,
  COUNT(*) AS calls,
  SUM(input_tokens) AS input_tokens,
  SUM(output_tokens) AS output_tokens,
  ROUND(SUM(estimated_cost_usd), 2) AS cost_usd
FROM inference_calls
WHERE occurred_at >= '2026-02-01'
  AND occurred_at < '2026-03-01'
GROUP BY feature
ORDER BY cost_usd DESC;

feature                    calls   input_tokens   output_tokens   cost_usd
-------------------------  ------  -------------  --------------  --------
document-summarization      60000     512,000,000      39,000,000  24100.00
support-bot                 84000      41,000,000      13,000,000   4100.00
structured-extraction       22000      88,000,000       6,000,000   3300.00
agent-lab                    7100      36,000,000      11,000,000   1800.00
general-chat                39000      18,000,000       7,000,000    700.00

The CLI wraps common reports:

# What drove February spend?
clawql inference spend \
  --from 2026-02-01 \
  --to 2026-03-01 \
  --group-by feature

# Who owns it?
clawql inference spend \
  --period month \
  --group-by team

# Which workspaces generated summarization cost?
clawql inference spend \
  --feature document-summarization \
  --group-by workspace_id \
  --period month \
  --limit 20

# Compare February to January.
clawql inference spend \
  --period month \
  --compare previous \
  --group-by feature

The comparison tells the story:

Feature                   Jan cost   Feb cost   Delta
------------------------  --------   --------   ------
document-summarization       $0.00  $24,100.00  +$24,100.00
support-bot              $3,900.00   $4,100.00     +$200.00
structured-extraction    $3,200.00   $3,300.00     +$100.00
agent-lab                  $900.00   $1,800.00     +$900.00
general-chat             $1,000.00     $700.00     -$300.00
Total                    $9,000.00  $34,000.00  +$25,000.00

The invoice total becomes an explanation: a new feature created 60,000 summarization calls and accounted for nearly all of the increase.

Lesson: a spend report is not a prettier invoice. It is a decomposition of spend into decisions owners can change.

The three-day spike

Monthly totals hide operational failures. The summarization launch had a specific shape.

SELECT
  date_trunc('day', occurred_at) AS day,
  feature,
  COUNT(*) AS calls,
  ROUND(SUM(estimated_cost_usd), 2) AS cost_usd
FROM inference_calls
WHERE occurred_at >= '2026-02-01'
  AND occurred_at < '2026-03-01'
  AND feature = 'document-summarization'
GROUP BY day, feature
ORDER BY day;

day          calls   cost_usd
----------   -----   --------
2026-02-01     410     144.20
2026-02-02     438     151.80
2026-02-03     421     149.60
...
2026-02-14   11204   4620.30
2026-02-15   13792   5891.10
2026-02-16    9488   3917.40
...
2026-02-28     780     272.60

The spike came from a deployment that re-summarized every document in a workspace after metadata changed. The agent loop retried failures with the full document attached. Three days produced more than $14,000 in spend.

The fix was not only “patch the loop.” It was:

Add idempotency keys for summarization jobs.
Cache summaries by document hash.
Cap workspace-level spend.
Set a virtual key budget for the documents team.
Alert at 75% and 90% of budget.
Add feature-level daily spend anomaly detection.

Lesson: billing surprises are usually time-local. If your attribution system only works at month end, it is too late to be operational.

Tier distribution and model escalation

Attribution should show model tier distribution, not just total cost.

clawql inference spend \
  --feature document-summarization \
  --period month \
  --group-by tier

Tier       Calls    Cost       Share of calls   Share of cost
--------   ------   --------   --------------   -------------
Frugal     42000    $3,600          70.0%           14.9%
Standard   13200    $6,500          22.0%           27.0%
Frontier    4800   $14,000           8.0%           58.1%

Only 8% of summarization calls hit Frontier, but they produced 58% of feature cost. That does not mean model escalation is bad. It means the expensive tail is where optimization should focus.

The team inspected escalation reasons:

clawql inference spend \
  --feature document-summarization \
  --group-by escalation_reason \
  --filter tier=frontier \
  --period month

escalation_reason                 calls   cost
--------------------------------  -----   -------
context_too_long                   2100   $6,900
low_confidence_summary             1300   $3,500
schema_validation_failed            900   $2,400
retry_after_tool_failure             500   $1,200

Now the fixes are specific:

context_too_long: chunk and summarize hierarchically.
low_confidence_summary: build a flywheel for routine summaries.
schema_validation_failed: improve prompt and schema constraints.
retry_after_tool_failure: fix the agent loop and idempotency.

Lesson: model escalation is only visible if tier is part of the attribution record. Without tier distribution, the expensive tail looks like generic provider spend.

Agent loops and BudgetExhausted

Agents need to treat budget as a runtime constraint, not a finance surprise.

def summarize_workspace(workspace_id: str, documents: list[Document], client):
    summaries = []

    for document in documents:
        try:
            summaries.append(
                client.summarize(
                    document=document,
                    headers={
                        "X-ClawQL-Workspace": workspace_id,
                        "X-ClawQL-Feature": "document-summarization",
                        "X-ClawQL-Operation": "summarize_document",
                    },
                )
            )
        except BudgetExhausted as exc:
            return {
                "status": "partial",
                "reason": "budget_exhausted",
                "workspace_id": workspace_id,
                "completed": len(summaries),
                "remaining": len(documents) - len(summaries),
                "budget": exc.budget_usd,
                "current_spend": exc.current_spend_usd,
            }

    return {"status": "complete", "summaries": summaries}

A well-behaved agent stops, summarizes partial progress, and asks for a budget decision. A badly behaved agent catches every exception and retries. The gateway should make the latter impossible by returning structured, non-retryable budget errors:

{
  "error": "budget_exhausted",
  "retryable": false,
  "team": "documents",
  "feature": "document-summarization",
  "message": "Monthly virtual key budget exhausted. Request budget increase or wait for reset."
}

The three-day February spike was partly an agent-loop problem. The loop treated summarization failures as transient and retried. Budget exhaustion should be a terminal condition unless a human explicitly raises the budget.

Lesson: budget is part of agent control flow. If agents cannot see and respect it, they will spend until an external system stops them.

Chargeback

Attribution changes organizational behavior when costs reach the teams that create them.

Chargeback does not have to mean internal invoices on day one. It can start as showback:

clawql inference spend \
  --period month \
  --group-by team,feature \
  --format csv \
  --output ./reports/inference-showback-2026-02.csv

team,feature,calls,cost_usd,budget_usd,utilization
documents,document-summarization,60000,24100.00,12000.00,200.8%
support,support-bot,84000,4100.00,4000.00,102.5%
operations,structured-extraction,22000,3300.00,5000.00,66.0%
experiments,agent-lab,7100,1800.00,500.00,360.0%

The report supports three conversations:

Budget ownership. The documents team owns summarization spend. If the feature is valuable, its budget should reflect that. If not, its rollout should slow down.

Unit economics. Cost per summarized document becomes a product metric. Teams can compare it to customer value, not just provider tokens.

Optimization priority. Teams with high utilization and high cost get engineering attention: caching, chunking, model escalation tuning, or flywheel training.

Chargeback should be paired with guardrails. If teams fear surprise bills, they will avoid useful AI features. If budgets and alerts are clear, spend becomes manageable.

Lesson: chargeback is not punishment. It is how inference spend becomes owned, forecastable, and optimizable.

Honest failure modes

Attribution systems fail in predictable ways.

Missing tags. If feature and operation are optional, high-volume callers will omit them. Virtual keys should provide defaults, and the gateway should reject production calls that cannot be attributed.

Wrong tags. Teams can mislabel calls accidentally or intentionally. Periodic reconciliation with service identity, deployment metadata, and workspace ownership helps catch drift.

Estimated cost mismatch. Budgets rely on estimated tokens and price tables before the provider returns final usage. Reconcile with actual invoices and tune estimates.

Shared workspaces. One workspace can represent multiple teams or customers. Decide whether chargeback follows the caller, workspace owner, or product feature.

Overly strict budgets. A budget gate that blocks critical workflows without escalation paths will be bypassed. Alerts, temporary increases, and emergency approval flows matter.

Attribution without action. Reports alone do not reduce spend. The system needs owners, budgets, optimization loops, and review cadence.

Privacy in reports. Workspace and user-level attribution can expose sensitive business activity. Reports should aggregate appropriately and respect access controls.

Lesson: attribution is a control system. Bad labels, stale price tables, and unactioned reports turn it back into accounting theater.

Implementation path

Day zero: route all production inference through the gateway. Raw provider keys should not be embedded in application services.

Week one: create virtual keys per team and feature. Set budgets even if they are generous at first. The key is ownership.

Week two: add request headers for workspace, feature, operation, and correlation ID. Make production calls fail closed when team or feature is missing.

Week three: enable the call store with the SQL schema above. Record estimated cost, tokens, model, tier, retry count, escalation reason, prompt hash, response hash, and policy manifest hash.

Month two: ship the first spend report: group by team, feature, operation, workspace, and tier. Compare the report to the provider invoice and reconcile price-table drift.

Month three: enable BudgetGate enforcement with alerts at 75% and 90%. Treat BudgetExhausted as non-retryable in agent loops.

Month four: add tier distribution review to the monthly cost meeting. Features with expensive Frontier tails become candidates for chunking, caching, prompt changes, or the flywheel.

Ongoing: run showback or chargeback monthly. Investigate any feature whose spend grows faster than usage. Review budgets before major launches. Drill into three-day spikes, not just monthly totals.

The February invoice did not need to be a surprise. A virtual key would have identified the documents team. Feature tags would have isolated summarization. The call store would have shown 60,000 calls. BudgetGate would have stopped the three-day spike. Tier distribution would have shown that Frontier was the expensive tail. Chargeback would have made the owner clear.

Lesson: the invoice tells you what left the bank account. Attribution tells you who spent it, what they got, whether the spend was allowed, and where to optimize before next month.

Product reference: docs.clawql.com/inference/clawql-inference (virtual keys and spend). Source: ClawQL on GitHub. Related: twelve layers of LLM cost, inference spend flywheel, model escalation.