When Rubrics Become Rewards

A mechanism win on a synthetic fixture answers one question: does the architecture work? It does not answer the question firms ask next: by how much, on a public scoreboard, with the same model on both arms? And it does not answer the compounding question: what do those scored sessions become after the report is filed?

If you treat evaluation as a one-shot score, the traces evaporate. If you treat evaluation as capital formation, every rubric criterion that fires is a training label.

This pairs with What Convergence Week Actually Proved (OpenBench A/B ledger + OpenBenchTrace wrapping RTP), The Session Nobody Started (event-triggered sessions that flush the same envelopes), Memory Finds. Ontology Decides., the API spend that never compounds, and Layer 12.

What is worth publishing before the numbers

Harvey’s Legal Agent Benchmark (LAB) is a large, expert-rubric scoreboard for legal agents. Our earlier work rebuilt the institutional-knowledge failure mode on a synthetic mini-firm: semantic memory can call the right tool and still score zero; typed predicates close the set. That is a mechanism claim. LAB is where you publish firm-facing numbers.

The interesting architecture is not the adapter wiring. It is the evaluation contract that makes those numbers — and the training data — honest:

Same model on both arms. ClawQL-on versus ClawQL-off only means something if the frontier model is identical. Mixing Sonnet on one arm and Opus on the other is not a comparison; it is a confounded story.
Same documents, different decision machinery. Pre-ingest matter files into a task-scoped vault so structured recall is available — without hiding a second corpus the baseline never sees.
Vault isolation between tasks. If matter A leaks into matter B’s recall, “wins” are contamination. Isolation is not ops hygiene; it is validity.
Score what the rubric scores. LAB uses expert criteria with all-pass semantics: every criterion must pass for a task score of 1.0. Criterion pass rate and all-pass rate answer different questions. Publish both. Do not blur a synthetic fixture with LAB.

Until those Opus numbers exist, the honest line is: mechanism proven, public scoreboard pending. Do not claim the scoreboard early.

Why firm-knowledge is the right family

Institutional questions are enumeration under constraints. Find every matter that meets a predicate. Reconstruct a client preference across history. Stop when the set is closed — not when the agent feels done.

That maps cleanly to LAB’s firm-knowledge family and to the failure Harvey already published: confident incompleteness. It also maps to the training question. The “right move” on these tasks is usually structured recall first, then targeted read — not fifty sequential file opens. You can see that move in a trace. You can reward it.

The trace is the product

Every LAB run (baseline and ClawQL) should leave a scored session record: task id, arm, turn sequence, tool calls, deliverable, criterion pass rate, all-pass, model, and consent scope for training use. Without the verdict fields, you have a transcript. With them, you have examples.

OpenBench already ships that shape as OpenBenchTrace wrapping RTP: the outer envelope carries arm / run / grader metadata; the inner six-node sequence (intent → retrieval → reasoning → execution → delta → verdict) is what SFT / DPO / GRPO formatters consume. Convergence Week packs with suitable_for_training: true are the frugal-model proof of the same flywheel — see What Convergence Week Actually Proved.

Two shapes matter immediately:

Passing ClawQL traces — high criterion pass rate, evidence of structured memory_recall with schema and filters — are supervised examples of the pattern you want a cheaper domain model to imitate.

Paired traces on the same task — ClawQL arm passes, baseline arm fails — are preference pairs. The prompt is shared. The difference is retrieval strategy and outcome. That is Direct Preference Optimization (DPO) fuel, not a blog anecdote.

If you also capture consent and provenance at run start, the same corpus can later feed a domain adapter without reconstructing history from screenshots.

Method choice follows the data, not the acronym

Fine-tuning method lists read like a menu. The useful framing is: what signal do you actually have?

Signal you have	Method that fits	Why
High-quality (prompt → good trajectory) examples	Supervised fine-tuning (SFT), usually via QLoRA on a single GPU	Establishes the basic pattern before preference pressure
Same task, better vs worse outcomes	DPO (or IPO on small pair sets; KTO when you only have good/bad labels, not clean pairs; ORPO when you want SFT+preference in one pass)	Teaches preference without a separate reward model
Automatically verifiable score per rollout	GRPO	Samples multiple completions, scores them with a reward function, trains on group-relative advantage — no critic model
Subjective quality that resists pairs and auto-scores	RLHF (reward model + PPO)	Justified when legal style is the target; usually overkill when the rubric already fires
Need self-monitoring / “search further” awareness	Constitutional critique → revision	Useful after the basic pattern exists
Two generations of the same model on the same tasks	SPIN (self-play DPO against the prior round)	The flywheel generates these pairs once you have sequential adapters

QLoRA is not a competing “method” in this table. It is the default how weights update on one GPU. DPO and GRPO choose the training signal; QLoRA (or LoRA on multi-GPU) chooses the adapter mechanics.

The length trap in preference data

DPO learns whatever differs between chosen and rejected. If you pair a three-turn structured-recall win against a thirty-turn baseline thrash, the model may learn “be shorter,” not “use typed filters.” Prefer pairs with similar structure and turn count where the difference is retrieval strategy and criterion outcome. A simple length-ratio guard on chosen vs rejected responses is boring infrastructure that prevents a real failure mode.

Why GRPO fits rubric benchmarks

GRPO wants a reward that fires without a human on every sample. Harvey-style criteria and completeness graders (exact set of matters, hard zero on false positives) are exactly that: deterministic enough to score rollouts during training.

A useful reward is not only “all criteria passed.” It can combine:

per-criterion F1 (completeness and precision),
an all-pass bonus,
a small bonus for the structured-recall tool pattern (schema + filters present),
a mild penalty for “semantic-only” recall when structured filters were available.

The model is then optimizing the metric the scoreboard cares about, while being nudged toward the retrieval move that produces it. You do not have to script the reasoning path turn by turn.

RLHF still has a place for memo tone and argument strength. For firm-knowledge enumeration, the rubric already is the reward model. Prefer the simpler loop.

A sequence that compounds

After a two-arm LAB sweep (or any rubric-scored agent corpus) lands:

Round 1 — SFT on strong ClawQL traces. Filter hard: high criterion pass rate, require tool evidence of structured recall. Goal: teach the pattern, not yet the preference.

Round 2 — DPO on ClawQL vs baseline pairs. Same tasks, opposite arms, length-balanced. Goal: prefer structured recall over sequential thrash.

Round 3 — GRPO against the rubric. Prompts from the task family; rollouts scored by criterion F1 (+ tool bonuses). Goal: maximize the scoreboard metric directly, building on the Round 2 adapter.

Round 4 — SPIN against the prior adapter. Current round’s traces vs previous round on the same tasks. Goal: iterative self-improvement until gains flatten.

Each round only earns promotion if held-out eval (same rubric family) clears a threshold. An adapter that looks clever in the training loss and regresses on firm-knowledge is not capital — it is a rollback candidate. Registration into the Frugal routing map is the last step, not the first; see model escalation for how cheap defaults stay safe.

What belongs in the evaluation design (and what does not)

Worth designing carefully before the expensive sweep:

Two arms, one model family for the published comparison.
Task-scoped memory so isolation is structural, not hopeful.
System guidance that states when to use structured recall versus full-text read — without turning the agent into a puppet that never has to choose.
Trace capture with arm, task, grader verdict, and training consent on every run.
Debug and harness correctness on a cheaper model; publish the scoreboard on the model peers will compare against. Judge models can stay cheaper than the agent model.

Not worth dumping into a public architecture essay: adapter source, sandbox startup scripts, dollar cost tables, workflow YAML. Those are product docs. The blog’s job is the decision structure — fair arms, trustworthy verdicts, method matched to signal, promotion gated by the same rubric.

Failure modes that poison both the score and the dataset

Contamination. Cross-task vault leak invents fake wins and teaches the wrong corpus.

Wrong-tool “wins.” Traces that guess the right answer without structured recall train the wrong habit. Adversarial filtering — keep only passes with verified tool evidence — keeps the supervised set sharp.

Over-ingestion / tool confusion. Too much noise in the vault or too many tools without clear roles can make the ClawQL arm worse. Those losses belong in the ledger. They are also useful rejected examples.

Confounded arms. Different models, different document visibility, or different turn budgets invalidate both the blog claim and the preference pairs.

Closing the loop

two-arm rubric sweep
  → scored traces (pass / fail / pairs)
  → SFT → DPO → GRPO → SPIN
  → domain adapter
  → Frugal routing for the next sweep
  → better traces

The ontology post showed why typed predicates close institutional sets. The spend-flywheel post showed why production verdicts must not evaporate. This is the hinge between them: design the benchmark so the verdict is real, then train on the move that earned it.

We will publish LAB criterion and all-pass rates when the Opus two-arm ledger exists. Until then, the claim is architectural: rubrics are not only a report. Done right, they are the reward function for the next model.