What Convergence Week Actually Proved

Ten-plus product claims. Live A/B benchmarks on GitHub Actions with a cheap OpenRouter model. On scores 1.0. Off scores 0.0. The ledger is public. The traces are not throwaway CI logs — each OpenBenchTrace pack wraps an RTP reasoning sequence the fine-tuning pipeline can already eat.

Here is what Convergence Week proved, how OpenBenchTrace and RTP fit together, and why that composition is the start of the flywheel — not the end of a demo week.

This pairs with When Rubrics Become Rewards, Memory Finds. Ontology Decides., The Session Nobody Started (Streams → RTP/OBT on production events), the API spend that never compounds, and J-Space / SGDOP. (Separate post: The Week Everything Converged — PorTAL / gRPC / OKF; different “convergence,” same summer.)

The setup

Most agent framework benchmarks use frontier models, curated inputs, and synthetic success criteria. That produces impressive numbers that evaporate in production.

OpenBench runs differently:

• Model: openrouter/deepseek/deepseek-chat — frugal tier, not Opus.
• Harness: OpenCode wired to clawql-inference.
• Hard caps: 50 turns, 180 seconds, 8,000 tokens per Ouroboros run (production-approximate failure modes, not infinity-context theater).
• Graders: score workspace artifacts plus required tool evidence. Both arms must produce real tool:clawql_* tool_use blocks to win — the off arm cannot score by generating plausible-looking prose about tools it never called.

A WIN means the ClawQL-on arm scores higher than ClawQL-off. 1.0 vs 0.0 means complete task success versus complete failure.

All results cite GitHub Actions run IDs. The living ledger is docs/benchmarks/openbench-results-ledger.md. OpenBench landed in PR #759.

What proved out

Ouroboros convergence

Task: ouroboros-oscillation-escape

Conflicting decoys try to send the agent into a loop. Without Ouroboros, the off arm thrashes for ~167 seconds and fails. With Ouroboros, the on arm converges in 5 turns and ~78 seconds.

In production document pipelines, agents hit the same shape: two validation rules disagree, two API responses contradict, a routing decision waits on missing info. Without convergence enforcement they burn tokens and either time out or emit garbage. With Ouroboros they exit in a defined turn budget.

Score: on 1.0, off 0.0. Replicated with doom_loop=deny and on a separate matrix. Runs: 30863572642, 30866904277, 30872913519.

The 78s vs 167s delta is inference cost. On thrash-prone tasks, Ouroboros roughly halves agent compute.

Vault memory

Three memory tasks, three decisive wins:

Headline runs include 31014040293 (roundtrip n=3) and 30872913516 / 30872437811.

Search-first, execute safety, audit, policy

The same frugal model that would guess or hallucinate calls without Panguard respects the policy boundary when it is active — behavioral proof of fail-closed, not a README claim.

Knowledge and operations wave

codegraph-guided-edit sits on new infrastructure — native TypeScript tree-sitter replacing the Python Graphify CLI path, 30+ language support (PR #793). The cell confirms agents use the codebase graph when ClawQL tools are present and fail without them.

composed-safe-rollout and several peers were later recollected as OpenBenchTrace v1.1 packs with RTP present and durable R2 sync (suitable_for_training: true on the on arm) — see 30985126247.

What came after the sprint (same harness)

Convergence Week was not the end of the ledger. On 2026-08-08 the same OpenBench design recorded a structured-ontology WIN on institutional-knowledge-enumerate (B-7.1): clawql-on 3/3 at 1.0, clawql-off 0.133, clawql-no-memory 0.0 — fair same-files, DeepSeek, taught filter shape not answer IDs (31255172649). That is the mechanism story in Memory Finds. Ontology Decides. with Actions receipts.

What is still honest about the limits

Multi-trial statistics. Many early cells were n=1. Phase 0 pushed several tasks to n=3. Directionally strong under a cheap model and hard caps is meaningful; it is not every cell at peer-review N yet.

Infrastructure noise. Matrices where OpenCode hung with no tool calls are logged as noise and excluded from claimed wins.

multi-provider-api-workflow. Early WIN (on 1.0 / off 0.75); treat as margin win until more trials lock the gap.

Confounded / gate-fail cells stay in the ledger. When an arm hid files, burned credits, or let bare grep compete unfairly, the ledger says Gate FAIL — those are not product wins. That honesty is part of the product.

Why cheap model, hard caps, hardened graders

When ClawQL tools take a DeepSeek model from 0.0 to 1.0, that is the product working — not the model. A claim that only holds on Opus or GPT-class frontier tells you about the model. OpenBench keeps the model constant and varies the tool surface so the delta is attributable to ClawQL.

Hard spend caps serve the same function. Enterprise pipelines do not get unlimited turns. 50 / 180s / 8k tokens are production-approximate: violating them in deployment is a failure mode.

Grader hardening matters most. Early graders could be fooled by off-arm prose that looked like tool calls. Current graders require real tool:clawql_* evidence in tool_use blocks.

OpenBenchTrace and RTP: outer envelope, inner reasoning

OpenBench does not only produce scores. It produces training capital — if the schemas compose.

OpenBenchTrace (OBT) is a collection schema. It answers: how do you capture a benchmark run — A/B arms, model, harness, grader verdict, spend-cap hits, PII scrub provenance — stably enough to publish and cite? It knows about run_id, arm, task_id, Actions IDs, clawql_version, verdict_source: grader.

RTP (Reasoning Trace Protocol) is a reasoning schema. It answers a different question: how do you capture what the agent did — intent, retrieval, reasoning chain, tool execution, state delta, verdict — with cryptographic consent provenance, domain-agnostic enough that an IDP session and a debugging session share one shape? It does not know about benchmark arms. It knows about world states and endogenous verification.

They compose cleanly: OBT wraps RTP. The outer envelope carries benchmark metadata. The inner turnSequence is RTP.

Tier 1 = deterministic graders (policy deny either blocked or it did not). Tier 2 = semantic graders. Designing OBT so messages / tool_calls serialize into a valid RTP turnSequence means every passing Convergence Week pack is simultaneously:

1. an OpenBench-specific benchmark record,
2. an RTP-compatible training trace,
3. (with consent scopes at job start) a dataset that can be licensed without retroactive cleanup.

Practical mapping details that matter for the flywheel:

• Consent token → RTP consentToken with community_model / dataset_licensing scopes, issued by the gateway at job start (OpenBench on your own infra still needs an explicit token if you want commercial reuse later).
• Grader verdict → RTP verdict with the right evaluatorTier.
• Merkle / turn hashing → RTP turn hash chaining so the pack is tamper-evident.

Publishing as OBT serves teams running OpenBench. Publishing as RTP-compatible serves any team building agent fine-tunes — whether or not they use ClawQL’s harness. A model trained on RTP across domains learns generalizable reasoning patterns, not only ClawQL tool spellings.

Same math, two jobs: NSV / SGDOP

RTP uses NSV and SGDOP as schema governance — keep domain extensions from being semantically redundant; find blind spots in reasoning coverage. ClawQL’s agent coordination layer uses the same mathematics as runtime swarm coordination — measure ensemble diversity; find blind spots in model coverage. When combined drift exceeds threshold, SGDOP points at the under-covered direction in embedding space. When RTP’s schema compiler evaluates a new node, SGDOP points at the under-covered direction in reasoning space. Recruit a model or add a schema node — same geometric question. Deeper treatment: J-Space, SGDOP, and Semantic Gradient Descent.

From ledger to flywheel

Convergence Week’s on/off pairs are already the training shapes the inference pipeline wants:

• Passing on-arm traces (CPR / grader 1.0, real tool evidence) → SFT.
• Same task, on wins / off fails → DPO chosen/rejected (length-balance the pairs or you teach verbosity).
• Verifiable grader scores → GRPO rewards when you optimize directly against the metric.

Promotion still gates on held-out eval before anything lands in tier-map.json for PAL routing. Method choice, length-ratio traps, Harvey rubric GRPO, and the SFT → DPO → GRPO → SPIN sequence are the subject of When Rubrics Become Rewards — this post is the proof that the eval side already produces the capital.

OpenBench A/B cell (frugal model, hard caps)
  → OBT envelope + RTP turnSequence → R2
  → filter / format by method
  → QLoRA adapter
  → eval gate
  → tier-map.json Frugal route
  → next cell uses a better cheap model

That is Layer 12 with receipts. Production event traffic emits the same envelopes via ClawQL Streams — Agent cells flush RTP/OBT on session close, not only CI cells.

What this means for evaluation

The Actions run IDs in this post are public. The harness is in-tree. The ledger updates when cells win, fail, or get invalidated.

Run OpenBench against your own workload. Point OpenCode at your ClawQL deployment. Keep the model constant. Require real tool evidence. If the on arm does not beat the off arm, the claim is not ready — no matter how good the blog draft looked.

Further reading

• Full run ledger — openbench-results-ledger.md
• OpenBench PR — #759 · Convergence follow-ons — #790–#796, #793 (codegraph)
• When Rubrics Become Rewards — SFT / DPO / GRPO / SPIN from scored traces
• Memory Finds. Ontology Decides. — B-7.1 structured predicate WIN
• The API Spend That Never Compounds · Twelve Layers of LLM Cost
• J-Space / SGDOP — RTP ↔ blind-spot geometry
• Ecosystem week (different topic) — The Week Everything Converged