The API Spend That Never Compounds

Every month you run production inference, you generate training data. Most teams let it evaporate. A practical guide to building the flywheel that turns API spend into proprietary model capital.

This pairs with Layer 12 of the twelve layers of LLM cost, model escalation, and the audit trail you can’t reconstruct — the flywheel only compounds when verdicts, redaction, provenance, and tier registration are part of the inference path.

The spend that evaporates

In January 2026, two operations teams started the same structured extraction project.

Both teams processed semi-structured business documents: onboarding forms, vendor packets, renewal notices, and PDF attachments that needed fields extracted into a clean schema. Both teams used a frontier model because the documents were messy and the cost of extraction errors was high. Both teams handled about 30,000 extraction calls per month.

Team A treated inference as a utility bill. The model returned JSON, the application consumed it, and the provider invoice arrived at the end of the month. When humans corrected bad extractions, those corrections stayed in the review UI. When downstream validation accepted good extractions, that signal stayed in the workflow database. The inference layer saw tokens and latency, not outcomes.

Team B treated inference as a data acquisition loop. Every extraction call wrote a structured record: prompt hash, response hash, schema, model, tier, cost, document type, validation result, human correction when present, and correlation_id. Passed calls became candidate training examples. Failed calls became evaluation cases. Corrected calls became high-value supervised examples after redaction.

By July 2026, the teams had different businesses.

Team A still paid for frontier extraction on routine documents. Their July bill looked like January with slightly more volume.

Team B had two flywheel cycles behind it. A custom Frugal model handled the high-volume document types. Frontier was reserved for ambiguous edge cases. Their July bill was lower than January despite higher traffic, and their model now encoded patterns specific to their documents, schemas, and reviewer preferences.

The difference was not that Team B “used a cheaper model.” The difference was that Team B converted production spend into proprietary training capital, then registered that capital back into production through model escalation.

Lesson: every production inference call is either a disposable expense or a candidate training example. The architecture decides which one it becomes.

Why most teams don’t capture it

The flywheel sounds obvious:

production calls
  → outcome-labeled examples
  → scrubbed training corpus
  → fine-tuned model
  → cheaper routine inference
  → more production calls
  → better examples

Most teams still do not build it because three requirements are missing.

Requirement 1: outcome signals. You need to know whether the model output was correct. Token logs do not tell you that. Provider invoices do not tell you that. A prompt and response without a verdict is not training data; it is a transcript.

Requirement 2: safe export. Production prompts and responses contain client names, account numbers, contract terms, health data, source code, and other sensitive material. Export has to scrub PII and preserve provenance before data leaves the operational store.

Requirement 3: production registration. A fine-tuned model that lives in a notebook or provider dashboard is not compounding. It has to become the default Frugal model for the task type, with fallback through model escalation when confidence drops.

Team A lacked all three. Team B built them before the first fine-tuning run.

Lesson: the flywheel is not “save prompts and fine-tune later.” It is verdict capture, safe export, and tier registration as one closed loop.

The flywheel log

The core object is a production inference record with an outcome.

{
  "id": "call_2026_03_14_000912",
  "correlation_id": "corr-doc-2026-03-14-87921",
  "team": "operations",
  "operation": "structured_extraction",
  "document_type": "vendor_onboarding_packet",
  "schema": "vendor_onboarding_v3",
  "model": "anthropic/claude-sonnet-4",
  "tier": "frontier",
  "input_tokens": 18420,
  "output_tokens": 1120,
  "estimated_cost_usd": 0.84,
  "prompt_hash": "sha256:0b25...",
  "response_hash": "sha256:6a91...",
  "cache_hit": false,
  "verdict": "passed",
  "verdict_source": "downstream_validation",
  "validator": "schema_and_erp_match",
  "created_at": "2026-03-14T17:22:09Z"
}

The verdict field is the hinge. Without it, the record is spend telemetry. With it, the record can enter the training pipeline.

A failed call is useful too:

{
  "id": "call_2026_03_14_000944",
  "correlation_id": "corr-doc-2026-03-14-87953",
  "operation": "structured_extraction",
  "document_type": "renewal_notice",
  "schema": "renewal_notice_v2",
  "tier": "frontier",
  "verdict": "failed",
  "verdict_source": "human_review",
  "failure_reason": "missed_auto_renewal_clause",
  "corrected_output_hash": "sha256:bd77..."
}

Passed calls train the model on correct behavior. Failed calls become holdout cases or corrected examples once a reviewer supplies the target output. Both matter, but they serve different stages of the loop.

Lesson: the flywheel starts with one extra field: verdict. Everything else depends on knowing whether production output was good.

Outcome signal approaches

Outcome signals do not have to be perfect on day one. They have to be explicit.

Team B started with downstream validation because the structured extraction output was already checked against schemas and ERP records. They later added human correction capture for fields that validators could not judge, such as non-standard termination clauses.

The export command filtered by signal:

# Passed production calls become training candidates.
clawql inference export \
  --operation structured_extraction \
  --verdict passed \
  --exclude-cache-hits \
  --min-date 2026-03-01 \
  --max-date 2026-03-31 \
  --format openai-jsonl \
  --output ./training/structured-extraction-2026-03.jsonl

# Failed and corrected calls become a separate supervised set.
clawql inference export \
  --operation structured_extraction \
  --verdict failed \
  --require-correction \
  --format openai-jsonl \
  --output ./training/structured-extraction-corrections-2026-03.jsonl

Cache hits are excluded because they do not represent fresh model behavior. They may be useful for replay tests, but they should not teach the next model to imitate stale cached outputs.

Lesson: verdict quality controls model quality. Start with the strongest signal you have, label its source, and keep weaker signals out of promotion-critical datasets until they are calibrated.

Presidio export

Production examples are sensitive. Team B’s export path ran PII detection and structured redaction before writing training files.

clawql inference export \
  --operation structured_extraction \
  --verdict passed \
  --exclude-cache-hits \
  --scrub-pii presidio \
  --format openai-jsonl \
  --output ./training/structured-extraction-2026-03-scrubbed.jsonl

The pipeline looked like this:

call store
  → verdict filter
  → cache-hit exclusion
  → Presidio detection
  → structured redaction
  → JSONL formatter
  → WORM TrainingLineage manifest

A scrubbed line preserves structure without leaking values:

{
  "messages": [
    {
      "role": "system",
      "content": "Extract fields into vendor_onboarding_v3 JSON."
    },
    {
      "role": "user",
      "content": "Vendor: [REDACTED:ORGANIZATION]. Tax ID: [REDACTED:US_TIN]. Contact: [REDACTED:EMAIL]. Payment terms: Net 45..."
    },
    {
      "role": "assistant",
      "content": "{\"vendor_name\":\"[REDACTED:ORGANIZATION]\",\"tax_id\":\"[REDACTED:US_TIN]\",\"payment_terms\":\"Net 45\"}"
    }
  ],
  "metadata": {
    "correlation_id": "corr-doc-2026-03-14-87921",
    "document_type": "vendor_onboarding_packet",
    "schema": "vendor_onboarding_v3",
    "verdict": "passed"
  }
}

The exact redaction policy depends on the domain. The important property is that export cannot bypass it. If data is not scrubbed and recorded in lineage, it is not eligible for training.

Lesson: PII scrubbing is not a notebook preprocessing cell. It is a gate in the export command, with provenance attached.

TrainingLineage WORM

Every dataset export needs a lineage record.

{
  "event_kind": "TRAINING_EXPORT",
  "export_id": "exp_structured_extraction_2026_03",
  "operation": "structured_extraction",
  "source_window": {
    "from": "2026-03-01T00:00:00Z",
    "to": "2026-03-31T23:59:59Z"
  },
  "filters": {
    "verdict": "passed",
    "exclude_cache_hits": true,
    "document_types": ["vendor_onboarding_packet", "renewal_notice"]
  },
  "source_records": 28734,
  "exported_records": 24108,
  "presidio_version": "2.2.354",
  "redaction_policy_hash": "sha256:8801...",
  "input_hash": "sha256:fa44...",
  "output_hash": "sha256:190d...",
  "sample_correlation_ids": ["corr-doc-2026-03-01-00102", "corr-doc-2026-03-14-87921"]
}

That record is written to WORM storage and chained with the same audit mechanism used for runtime decisions. The point is not bureaucracy. It is defensibility.

When someone asks, “What data trained structured-extraction-v2?” the answer should be a lineage chain:

TRAINING_EXPORT exp_structured_extraction_2026_03
  → PRESIDIO_SCRUB redaction_policy sha256:8801...
  → FINE_TUNE_JOB ft_structured_extraction_v1
  → BENCHMARK_RESULT bench_structured_extraction_v1
  → TIER_REGISTRATION frugal structured_extraction

No lineage, no production registration.

Lesson: the proprietary model asset is only defensible if you can prove what went into it, how it was scrubbed, and why it was promoted.

Fine-tuning economics

Team A and Team B spent similarly in January and February because both were collecting production volume on frontier models.

The first cycle did not eliminate frontier calls. It moved routine structured extraction to a custom Frugal model and escalated uncertain cases.

After two cycles, the custom Frugal model knew the team’s schema conventions: which fields were mandatory, how reviewers represented missing values, how renewal clauses appeared in their documents, and which vendor-packet sections were irrelevant. That knowledge did not come from a synthetic benchmark. It came from production traffic with verdicts.

The economics improved because the distribution changed. The frontier model still mattered, but it was no longer the default for routine extraction.

Lesson: fine-tuning pays when it changes tier distribution. The win is not a cheaper call in isolation; it is moving routine volume to a custom Frugal baseline.

Tier registration via model escalation

The trained model has to enter the gateway’s tier map. Team B registered the second-cycle model as the Frugal default for structured extraction.

{
  "policyVersion": "2026.07.01",
  "modelEscalation": {
    "enabled": true,
    "tiers": {
      "frugal": {
        "default": "local/structured-extraction-v2",
        "maxCostPer1kTokens": 0.0004
      },
      "standard": {
        "default": "groq/llama-3.3-70b",
        "maxCostPer1kTokens": 0.002
      },
      "frontier": {
        "default": "anthropic/claude-sonnet-4",
        "maxCostPer1kTokens": 0.015
      }
    },
    "domainRoutes": {
      "structured_extraction": {
        "defaultTier": "frugal",
        "frugalModel": "local/structured-extraction-v2",
        "minConfidence": 0.86,
        "escalateOn": ["schema_validation_failed", "low_confidence", "policy_conflict"]
      }
    }
  }
}

# tier-map.json is reviewed, hashed, and applied as a policy manifest.
clawql inference policy apply ./tier-map.json

# Verify production routing for the task type.
clawql inference policy route \
  --operation structured_extraction \
  --document-type vendor_onboarding_packet

Model escalation then runs the loop:

1. Start structured extraction on the custom Frugal model.
2. Validate output against the schema and downstream systems.
3. Escalate to Standard when confidence or validation fails.
4. Escalate to Frontier only for the hardest cases.
5. WORM-log the routing decision and verdict.
6. Export passed Frugal calls for the next cycle.

Credit where it’s due: the “start cheap, escalate on failure” idea comes from Q00/ouroboros PAL work. ClawQL’s model escalation applies that principle inside a gateway with WORM-logged transitions, verdict-aware export, and policy-manifest tier registration.

Lesson: a fine-tuned model compounds only after it becomes the production Frugal route for its task. Model escalation is the mechanism that keeps quality from falling while the distribution shifts cheaper.

The switching cost

The flywheel creates a switching cost, but not the usual kind.

The custom Frugal model is trained on:

• The organization’s document types.
• The organization’s schemas.
• The organization’s corrections.
• The organization’s validation rules.
• The organization’s redaction policy.
• The organization’s WORM lineage.

If Team B moves inference traffic to another gateway that does not preserve verdicts, exports, manifests, and tier registration, the flywheel stops. They can move the model artifact if licensing permits. They cannot recreate the accumulated lineage and routing history unless the new system carries the same records.

That switching cost is not a dark pattern. It is capital formation. Value naturally accrues where production calls, verdicts, and training lineage accumulate.

This is why Layer 12 is different from caching or prompt trimming. Layers 1–11 reduce current cost. The flywheel changes future cost by building an asset.

Lesson: the defensible asset is not merely the fine-tuned weights. It is the chain of verified production examples, redaction manifests, benchmark results, and tier registrations that made those weights safe to use.

Observability for the flywheel

You should be able to see whether the flywheel is spinning.

# Is Frugal handling more structured extraction over time?
clawql inference spend \
  --operation structured_extraction \
  --group-by tier \
  --period month

# Are verdict rates stable after registration?
clawql inference export \
  --operation structured_extraction \
  --verdict passed \
  --count-only \
  --period week

clawql inference export \
  --operation structured_extraction \
  --verdict failed \
  --count-only \
  --period week

# What training lineage produced the current Frugal model?
clawql inference lineage show local/structured-extraction-v2

The dashboard should show:

If Frugal share rises but verdict pass rate falls, the model was promoted too aggressively. If verdict rate is stable but Frontier share does not fall, escalation thresholds may be too conservative. If corpus size grows but benchmark quality stalls, the dataset may be too noisy or too narrow.

Lesson: the flywheel is an operational system. Measure distribution, quality, lineage, and cost together.

Honest failure modes

The flywheel is powerful, but it is not automatic magic.

Cold start. A new task type with no verified examples cannot produce a useful custom model. Plan for one or two months of collection before the first serious training run.

Noisy verdicts. If “workflow completed” is treated as equivalent to “model was correct,” the dataset can accumulate silent errors. Separate strong signals from weak signals and benchmark before registration.

PII leakage. Exporting production data without structural redaction creates compliance risk. The export command should fail closed when redaction policy is missing.

Domain drift. A model trained on January vendor packets may degrade when July packets use a new template. Model escalation catches some of this by escalating failures, but monitoring has to catch the drift.

Out-of-domain routing. A structured-extraction model can perform badly on legal synthesis or support chat. Domain routes need task classification, and misrouted calls should not enter the training corpus.

Provider lock-in. Cloud fine-tuned models may be hard to move. Local fine-tuning improves portability but requires GPU operations. Choose deliberately.

Training data poisoning. If upstream document ingest admits adversarial or poisoned content, the flywheel can amplify it. Ingest scanning and audit provenance are prerequisites, not optional extras.

Lesson: the flywheel compounds whatever your production process emits. If the process emits clean, verified examples, value compounds. If it emits noise, risk compounds.

Implementation path

Day zero: enable the inference call store. Capture operation, document type, schema, model, tier, cost, prompt hash, response hash, cache status, and correlation_id.

Week one: add verdicts. Start with downstream validation for structured extraction: schema validation, ERP match, required-field checks, and human approval when available.

Week two: run a dry export. Filter to verdict=passed, exclude cache hits, scrub with Presidio, and inspect the JSONL manually. Count examples by document type.

Month two: write TrainingLineage WORM records for every export. Include source window, filters, redaction policy hash, input hash, output hash, and sample correlation IDs.

Month three: fine-tune the highest-volume task type. Benchmark against the current Frugal and Standard models on held-out examples. Do not register the model unless it improves cost without unacceptable quality loss.

Month four: register the model in tier-map.json as the Frugal default for that operation. Enable model escalation to Standard and Frontier on low confidence or validation failure.

Month five: repeat the export with production calls handled by the custom Frugal model. Train the second cycle. Compare tier distribution before and after registration.

Ongoing: review flywheel metrics monthly: corpus growth, pass rate, escalation rate, Frontier share, cost per document, and lineage verification. Start new flywheels per task type, not globally.

Team A paid for inference seven times from January through July and ended with invoices. Team B paid for inference seven times and ended with a cheaper production path plus a proprietary extraction model backed by lineage. Same starting point. Different architecture.

Lesson: API spend compounds only if you close the loop. Production calls need verdicts, exports need redaction and lineage, fine-tuned models need benchmarks, and the gateway needs model escalation to put the new Frugal model to work.

---

Reference implementation: docs.clawql.com/architecture/token-efficiency (Layer 12). Source: ClawQL on GitHub. Related: twelve layers of LLM cost, model escalation, audit trail reconstruction.