The invoice says you spent $34,000 on LLM inference last month. It doesn’t say who, what, or why. A practical guide to building the attribution system that turns a billing line item into an operational signal.
This pairs with Layer 9 of the twelve layers of LLM cost, the API spend flywheel, model escalation, and the audit trail you can’t reconstruct.
The Bill That Surprised Everyone
In February 2026, an operations platform opened its LLM provider invoice and saw $34,000 in inference spend. January had been $9,000.
The product team had expected an increase. They had launched a document summarization feature for customer workspaces — the feature took uploaded PDFs, summarized the contents, extracted action items, and wrote a short explanation back into the workspace activity feed. What nobody expected was the scale. The summarization feature had made 60,000 calls in February. Some calls summarized short memos. Others summarized 200-page packets. Some workspaces triggered summarization every time a file changed. A few agent loops retried failed summaries with the full document context again and again.
The provider dashboard showed spend by model and day. It did not answer the questions leadership asked: which team owned the 60,000 calls, which customer workspaces drove the spike, which feature path was responsible, how much came from routine summarization versus retries, why a few calls cost several dollars each, whether March would be $34,000 again or whether February was an anomaly.
The team spent a week joining provider exports, application logs, and deployment timelines. They eventually found the culprit: a three-day spike after document summarization launched with no per-team virtual key, no feature-level attribution, and no budget gate. The invoice was accurate. It just wasn’t useful.
The provider invoice is a receipt. Attribution is what turns the receipt into an operational signal.
Why Provider Dashboards Stop Short
Provider dashboards answer provider questions: how many tokens were billed, which model was used, and what the invoice total is. Production teams need different dimensions.
| Question | Provider dashboard | Attribution system |
|---|---|---|
| Who spent the money? | Sometimes API key | Team, service, workspace, virtual key |
| What feature spent it? | No | Operation and feature tags |
| Why did it happen? | No | Correlation ID, trace, retry reason, model escalation |
| Was it valuable? | No | Verdict and downstream outcome |
| Could it have been prevented? | No | Budget gate, rate limit, policy |
| What should change? | No | Tier distribution and optimization loop |
API keys operate at the wrong granularity. A shared key used by three services creates one billing line. Model names don’t help either — “$18,000 on Claude” doesn’t tell you whether the spend came from summarization, support chat, extraction, or a runaway agent. Token counts are similarly opaque. A 50,000-token call might be legitimate legal synthesis or a loop that re-sent the same PDF five times.
Time aggregation compounds the problem. February’s $34,000 total hid a three-day period where the new summarization workflow consumed most of its budget before anyone looked at the dashboard.
Attribution needs dimensions the provider cannot know: team, feature, operation, workspace, verdict, retry, budget, and tier.
The Virtual Key Pattern
The first fix is to stop sharing raw provider keys across teams and features. Every caller should use a virtual key issued by the gateway.
clawql inference keys create \
--team documents \
--feature document-summarization \
--budget 12000 \
--period monthly \
--default-operation summarize_document
clawql inference keys create \
--team support \
--feature support-bot \
--budget 4000 \
--period monthly \
--default-operation answer_support_question
clawql inference keys create \
--team experiments \
--feature agent-lab \
--budget 500 \
--period monthly \
--default-operation experimental_agentApplications call the gateway with the virtual key:
from openai import OpenAI
client = OpenAI(
base_url="https://inference.example.com/v1",
api_key="vk_documents_2026_02",
)
response = client.chat.completions.create(
model="clawql:auto",
messages=[{"role": "user", "content": summarization_prompt}],
extra_headers={
"X-ClawQL-Workspace": "workspace_481",
"X-ClawQL-Feature": "document-summarization",
"X-ClawQL-Operation": "summarize_document",
"X-ClawQL-Correlation-Id": correlation_id,
},
)The gateway maps virtual keys to defaults:
{
"virtual_key_id": "vk_documents_2026_02",
"team": "documents",
"feature": "document-summarization",
"default_operation": "summarize_document",
"budget_usd": 12000,
"period": "monthly",
"allowed_environments": ["production"],
"rate_limit_rpm": 600
}If the application forgets to send a team header, the virtual key supplies one. If it omits feature or operation, defaults apply. Explicit request headers can refine attribution, but the call is never anonymous.
Attribution starts at the credential boundary. A raw provider key tells you the provider account. A virtual key tells you the operational owner.
BudgetGate
Attribution after the fact is useful. Prevention is different.
BudgetGate runs before the gateway forwards a request to the provider. It estimates the call cost, checks the virtual key’s remaining budget, reserves spend, and fails closed when the budget is exhausted.
class BudgetExhausted(Exception):
def __init__(self, *, team, feature, budget_usd,
current_spend_usd, estimated_cost_usd):
self.team = team
self.feature = feature
self.budget_usd = budget_usd
self.current_spend_usd = current_spend_usd
self.estimated_cost_usd = estimated_cost_usd
class BudgetGate:
def __init__(self, budget_store, price_table):
self.budget_store = budget_store
self.price_table = price_table
def check_and_reserve(self, *, virtual_key, model,
estimated_input_tokens, estimated_output_tokens):
estimated_cost = self.price_table.estimate(
model=model,
input_tokens=estimated_input_tokens,
output_tokens=estimated_output_tokens,
)
current_spend = self.budget_store.current_spend(
virtual_key_id=virtual_key.id,
period=virtual_key.period,
)
if current_spend + estimated_cost > virtual_key.budget_usd:
raise BudgetExhausted(
team=virtual_key.team,
feature=virtual_key.feature,
budget_usd=virtual_key.budget_usd,
current_spend_usd=current_spend,
estimated_cost_usd=estimated_cost,
)
self.budget_store.reserve(
virtual_key_id=virtual_key.id,
amount_usd=estimated_cost,
)
return estimated_costThe caller receives a structured error:
{
"error": "budget_exhausted",
"team": "documents",
"feature": "document-summarization",
"budget_usd": 12000.0,
"current_spend_usd": 11987.42,
"estimated_cost_usd": 18.31,
"period": "2026-02",
"resets_at": "2026-03-01T00:00:00Z"
}For the February spike, a $12,000 budget would have stopped the summarization feature before it reached $34,000. A 75% alert would have fired during the three-day spike. A per-workspace rate limit would have caught the loop even earlier.
The Call Store Schema
The call store is the source of truth for attribution — append-only operational telemetry, not a monthly spreadsheet.
CREATE TABLE inference_calls (
id UUID PRIMARY KEY,
occurred_at TIMESTAMPTZ NOT NULL,
correlation_id TEXT NOT NULL,
request_id TEXT NOT NULL,
team TEXT NOT NULL,
feature TEXT NOT NULL,
operation TEXT NOT NULL,
workspace_id TEXT,
virtual_key_id TEXT NOT NULL,
environment TEXT NOT NULL,
provider TEXT NOT NULL,
model TEXT NOT NULL,
tier TEXT NOT NULL CHECK (tier IN ('frugal', 'standard', 'frontier')),
input_tokens INTEGER NOT NULL,
output_tokens INTEGER NOT NULL,
estimated_cost_usd NUMERIC(12, 6) NOT NULL,
actual_cost_usd NUMERIC(12, 6),
latency_ms INTEGER NOT NULL,
cache_hit BOOLEAN NOT NULL DEFAULT false,
retry_count INTEGER NOT NULL DEFAULT 0,
escalation_from TEXT,
escalation_reason TEXT,
verdict TEXT CHECK (verdict IN ('passed', 'failed', 'hitl')),
prompt_hash TEXT NOT NULL,
response_hash TEXT,
policy_manifest_hash TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX inference_calls_time_idx ON inference_calls (occurred_at);
CREATE INDEX inference_calls_team_period_idx ON inference_calls (team, occurred_at);
CREATE INDEX inference_calls_feature_period_idx ON inference_calls (feature, occurred_at);
CREATE INDEX inference_calls_correlation_idx ON inference_calls (correlation_id);The fields that matter most operationally are team, feature, operation, workspace_id, virtual_key_id, tier, estimated_cost_usd, retry_count, and correlation_id. The provider invoice can be reconciled later with actual_cost_usd. The gateway needs estimated_cost_usd immediately so budgets and alerts can work before the invoice arrives.
If the call store lacks attribution dimensions, the finance team will recreate them later with guesswork. Capture them at the gateway instead.
Spend Reports
Once calls are tagged, the February invoice becomes queryable.
SELECT
feature,
COUNT(*) AS calls,
SUM(input_tokens) AS input_tokens,
SUM(output_tokens) AS output_tokens,
ROUND(SUM(estimated_cost_usd), 2) AS cost_usd
FROM inference_calls
WHERE occurred_at >= '2026-02-01'
AND occurred_at < '2026-03-01'
GROUP BY feature
ORDER BY cost_usd DESC;feature calls input_tokens output_tokens cost_usd
------------------------- ------ --------------- -------------- --------
document-summarization 60000 512,000,000 39,000,000 24100.00
support-bot 84000 41,000,000 13,000,000 4100.00
structured-extraction 22000 88,000,000 6,000,000 3300.00
agent-lab 7100 36,000,000 11,000,000 1800.00
general-chat 39000 18,000,000 7,000,000 700.00The CLI wraps common reports:
# What drove February spend?
clawql inference spend \
--from 2026-02-01 --to 2026-03-01 \
--group-by feature
# Which workspaces generated summarization cost?
clawql inference spend \
--feature document-summarization \
--group-by workspace_id \
--period month --limit 20
# Compare February to January
clawql inference spend \
--period month --compare previous \
--group-by featureThe comparison tells the story immediately:
Feature Jan cost Feb cost Delta
------------------------ -------- -------- ----------
document-summarization $0.00 $24,100.00 +$24,100.00
support-bot $3,900.00 $4,100.00 +$200.00
structured-extraction $3,200.00 $3,300.00 +$100.00
agent-lab $900.00 $1,800.00 +$900.00
general-chat $1,000.00 $700.00 -$300.00
Total $9,000.00 $34,000.00 +$25,000.00The $25,000 increase is almost entirely one new feature. That answer takes seconds to produce from the call store. It took the team a week to reconstruct from provider exports.
A spend report is a decomposition of spend into decisions owners can change. It’s not a prettier invoice.
The Three-Day Spike
Monthly totals hide operational failures. The summarization launch had a specific shape:
SELECT
date_trunc('day', occurred_at) AS day,
COUNT(*) AS calls,
ROUND(SUM(estimated_cost_usd), 2) AS cost_usd
FROM inference_calls
WHERE occurred_at >= '2026-02-01'
AND occurred_at < '2026-03-01'
AND feature = 'document-summarization'
GROUP BY day
ORDER BY day;day calls cost_usd
---------- ----- --------
2026-02-01 410 144.20
2026-02-02 438 151.80
...
2026-02-14 11204 4620.30
2026-02-15 13792 5891.10
2026-02-16 9488 3917.40
...
2026-02-28 780 272.60Three days produced more than $14,000. The spike came from a deployment that re-summarized every document in a workspace after metadata changed. The agent loop retried failures with the full document attached each time.
The fixes were specific: add idempotency keys for summarization jobs, cache summaries by document hash, cap workspace-level spend, set a virtual key budget for the documents team, alert at 75% and 90% of budget, and add feature-level daily spend anomaly detection. Patching the loop was necessary but not sufficient — the budget gate would have stopped the bleeding before anyone read the daily report.
Billing surprises are usually time-local. Attribution that only works at month end arrives too late.
Tier Distribution and Model Escalation
Attribution should show model tier distribution alongside total cost.
clawql inference spend \
--feature document-summarization \
--period month \
--group-by tierTier Calls Cost Share of calls Share of cost
-------- ------ -------- -------------- -------------
Frugal 42000 $3,600 70.0% 14.9%
Standard 13200 $6,500 22.0% 27.0%
Frontier 4800 $14,000 8.0% 58.1%Only 8% of summarization calls hit Frontier, but they accounted for 58% of feature cost. The expensive tail is where optimization should focus.
Inspecting escalation reasons makes the fixes concrete:
clawql inference spend \
--feature document-summarization \
--group-by escalation_reason \
--filter tier=frontier \
--period monthescalation_reason calls cost
-------------------------------- ----- -------
context_too_long 2100 $6,900
low_confidence_summary 1300 $3,500
schema_validation_failed 900 $2,400
retry_after_tool_failure 500 $1,200context_too_long means the chunking strategy is wrong. low_confidence_summary is a flywheel training opportunity. schema_validation_failed points to prompt and schema constraints that need work. retry_after_tool_failure goes back to the agent loop and idempotency problem.
Model escalation is invisible without tier in the attribution record. Without it, the expensive Frontier tail looks like generic provider spend and gets no specific attention.
Agents and BudgetExhausted
Agents need to treat budget as a runtime constraint, not a finance problem.
def summarize_workspace(workspace_id: str, documents: list, client):
summaries = []
for document in documents:
try:
summaries.append(
client.summarize(
document=document,
headers={
"X-ClawQL-Workspace": workspace_id,
"X-ClawQL-Feature": "document-summarization",
"X-ClawQL-Operation": "summarize_document",
},
)
)
except BudgetExhausted as exc:
return {
"status": "partial",
"reason": "budget_exhausted",
"workspace_id": workspace_id,
"completed": len(summaries),
"remaining": len(documents) - len(summaries),
"budget": exc.budget_usd,
"current_spend": exc.current_spend_usd,
}
return {"status": "complete", "summaries": summaries}A well-behaved agent stops, reports partial progress, and surfaces a budget decision. The gateway reinforces this by returning non-retryable budget errors:
{
"error": "budget_exhausted",
"retryable": false,
"team": "documents",
"feature": "document-summarization",
"message": "Monthly virtual key budget exhausted. Request budget increase or wait for reset."
}The February spike was partly an agent-loop problem because the loop treated summarization failures as transient and retried. Budget exhaustion should be a terminal condition unless a human explicitly raises the budget.
Chargeback
Attribution changes organizational behavior when costs reach the teams that create them. Chargeback doesn’t have to mean internal invoices on day one — it can start as showback:
clawql inference spend \
--period month \
--group-by team,feature \
--format csv \
--output ./reports/inference-showback-2026-02.csvteam,feature,calls,cost_usd,budget_usd,utilization
documents,document-summarization,60000,24100.00,12000.00,200.8%
support,support-bot,84000,4100.00,4000.00,102.5%
operations,structured-extraction,22000,3300.00,5000.00,66.0%
experiments,agent-lab,7100,1800.00,500.00,360.0%The report supports three conversations. Budget ownership: the documents team owns summarization spend, and if the feature is valuable its budget should reflect that. Unit economics: cost per summarized document becomes a product metric teams can compare to customer value rather than provider tokens. Optimization priority: teams with high utilization and high cost become candidates for engineering attention — caching, chunking, model escalation tuning, or flywheel training.
Chargeback should be paired with guardrails. Teams that fear surprise bills will avoid useful AI features. Teams with clear budgets and alerts can manage spend without fear.
Honest Failure Modes
Missing tags. If feature and operation are optional, high-volume callers will omit them. Virtual keys should provide defaults, and the gateway should reject production calls that can’t be attributed.
Wrong tags. Teams can mislabel calls accidentally. Periodic reconciliation with service identity, deployment metadata, and workspace ownership catches drift.
Estimated cost mismatch. Budgets rely on estimated tokens and price tables before the provider returns final usage. Reconcile with actual invoices and tune estimates.
Shared workspaces. One workspace can represent multiple teams or customers. Decide whether chargeback follows the caller, workspace owner, or product feature before the first showback report.
Overly strict budgets. A budget gate that blocks critical workflows without escalation paths will be bypassed. Alerts, temporary increases, and emergency approval flows matter as much as the gate itself.
Attribution without action. Reports alone don’t reduce spend. The system needs owners, budgets, optimization loops, and a review cadence.
Implementation Path
Start by routing all production inference through the gateway. Raw provider keys should not be embedded in application services.
Create virtual keys per team and feature. Set budgets even if they’re generous at first — ownership matters more than precision in week one. Add request headers for workspace, feature, operation, and correlation ID. Make production calls fail closed when team or feature is missing.
Enable the call store with the schema above. Record estimated cost, tokens, model, tier, retry count, escalation reason, prompt hash, response hash, and policy manifest hash. Ship the first spend report once a month of data has accumulated — group by team, feature, operation, workspace, and tier, then reconcile against the provider invoice to catch price-table drift.
Enable BudgetGate enforcement with alerts at 75% and 90%. Treat BudgetExhausted as non-retryable in agent loops. Once budgets are in place, add tier distribution review to the monthly cost meeting. Features with expensive Frontier tails become candidates for chunking, caching, prompt changes, or the flywheel.
Run showback or chargeback monthly. Investigate any feature whose spend grows faster than usage. Review budgets before major launches. Drill into three-day spikes rather than monthly totals.
The February invoice was retrievable from a call store that had team, feature, workspace, tier, retry count, and escalation reason on every record. A virtual key would have identified the documents team on day one. BudgetGate would have stopped the three-day spike mid-day. Tier distribution would have surfaced the expensive Frontier tail immediately. None of it required a week of manual reconstruction.
Product reference: docs.clawql.com/inference/clawql-inference. Source: ClawQL on GitHub. Related: twelve layers of LLM cost, inference spend flywheel, model escalation.