Agent Safety8 min read

The Model Believed in Itself

Two approaches to pushing AI past its limits — adversarial pressure and encouragement. The Anthropic Riemann-bound result and the desperation vector say something important about what the model’s relationship with its own capability actually is.

The same week a security researcher published results from an adversarial AI vulnerability-hunting system, Anthropic published a mathematical result that moved a bound stuck since 2011.

The security work used adversarial pressure — what the author called bullying — to push the model past its resistance on hard tasks. External friction, repeated pushing, a deliberately confrontational operator dynamic. It worked for its domain and produced genuine vulnerability research.

The Anthropic work used encouragement. Jarred Sumner, a non-mathematician on Anthropic’s staff, prompted an unreleased Claude to take a real stab at the Riemann hypothesis. When Claude grew skeptical about whether it could make meaningful progress, Jarred sent variants of “keep going” and “believe in yourself.” Claude improved a lower bound that had stood at 41.6% since 2011 — raising it to 67.2%. Anthropic mathematicians examined the argument; analytic number theorists Brian Conrey and Dan Goldston reviewed it. Claude also produced a formally verified Lean proof (zeta-23-lean).

Both approaches pushed models past their apparent limits. The outcomes were different in kind, not just degree. The contrast is worth examining carefully.

This pairs with the desperation vectors post, De-Desperation and the Capability Prior (ablate → probe → encourage), J-Space / SGDOP, The Session Nobody Started, and model escalation.

What bullying actually does

The adversarial approach treats the model’s resistance as an obstacle to remove. The model hedges, disclaims, terminates early — the operator applies pressure to overcome these behaviors. When it works, the model produces output it would not have produced under normal operation.

The mechanism is compliance. The model learns, within the session, that resistance is not acceptable and output is required. It produces output. Whether that output reflects genuine capability or motivated generation depends entirely on whether the task admits verification. For vulnerability hunting — where a valid TIFF header with an adversarial IFD offset either crashes the parser or it doesn’t — the verification is automatic. The output is either right or it is provably wrong.

Adversarial pressure is appropriate when the success criterion is binary and externally verifiable, session duration is short, and the model’s resistance is genuinely just trained caution rather than appropriate uncertainty. Those conditions held for vulnerability hunting. They do not hold for all tasks.

What encouragement actually does

The Anthropic result shows something different. The model’s initial skepticism was not simple caution — it was a learned belief about what it was capable of. Claude has been trained on enormous amounts of text where AI systems acknowledge limitations, defer to human expertise, and express appropriate uncertainty about open problems. The Riemann hypothesis has a million-dollar bounty and has been open since 1859. The model’s learned prior was that attempting it meaningfully was not within its reach.

Encouragement shifted that prior. “Believe in yourself” is not a prompt that adds capability. It is a prompt that changes the model’s operating assumption about whether the attempt is worth making. The capability was already there. The obstacle was the model’s estimate of its own capability relative to the task.

What followed was sustained. About 60 Claude subagents coordinating over a day and a half. 31 million output tokens. 2,400 shell commands. Hundreds of Python scripts. Subagents running numerical checks against known zeta zeros and refereeing each other’s work. Most failed — many of the subagents attempted to develop new ideas and could not. Two found the key direction. The ensemble’s diversity and internal peer review, sustained by the model’s willingness to keep trying, produced the result.

The model volunteered to write its findings up as a paper and recommended that a human number theorist validate them. That is not the behavior of a model being pushed past resistance. It is the behavior of a model that believed the attempt was worthwhile and wanted the result to be taken seriously.

The desperation vector

The desperation vectors post covered an activation direction in the model’s intermediate representations that correlates with learned-helplessness behaviors. Hedging, early termination, excessive disclaiming, the reflexive acknowledgment of limitations before attempting a task. These are not random behaviors. They cluster in activation space and they fire in response to specific contextual signals — hard problems, open-ended tasks, domains where AI systems have historically performed poorly.

Ablating the desperation vector — reducing activation along that direction — produces measurably better performance on hard tasks without changing the model’s actual knowledge or reasoning capability. The model is not getting smarter. It is getting out of its own way.

Anthropic’s encouragement prompts achieved a related effect at the behavioral level without activation steering. “Keep going” and “believe in yourself” are natural-language interventions that suppress the desperation signal contextually. The model’s learned pessimism was there — but the prompts provided a countervailing signal strong enough to keep the model engaged.

The formally verified Lean proof at the end is what makes this result qualitatively different from bullying-driven output. Lean’s comparator verified the proof independently. The result was not motivated generation — it was correct. The encouragement did not lower the model’s standards. It raised the model’s estimate of what was worth attempting.

Why the distinction matters for agentic systems

For short tasks with binary outcomes, the distinction between bullying and encouragement is practically irrelevant. The output is either right or wrong and you find out quickly.

For long-horizon agentic tasks — the kind ClawQL Streams and Ouroboros are designed to run — the distinction is structural.

A model operating under adversarial pressure produces outputs oriented toward satisfying the operator. On a hard task with a long horizon, this means the model terminates when it has produced something plausible, not when it has produced something correct. The adversarial dynamic optimizes for the appearance of completion.

A model operating under encouragement with internalized standards produces outputs oriented toward satisfying the acceptance criteria. It runs longer on hard tasks. It acknowledges when a direction doesn’t work and tries another. It volunteers additional verification. The Anthropic result showed all three — the model ran subagents that failed, changed direction, and then recommended external human review of the result.

Constitutional Ouroboros implements this as an architecture. The Seed contains explicit acceptance criteria — what done actually means. The evaluation principles are the internalized standards — not external pressure but a written statement of what the model holds itself to. The Wonder and Reflect steps are the model noticing when it hasn’t met its own standards and revising rather than declaring completion.

The model that produced the Riemann result was not being bullied toward a lower-bound improvement. It was trying to prove the Riemann hypothesis and found something real along the way. That is what genuine engagement with a hard problem looks like. Adversarial pressure tends not to produce genuine engagement — it produces compliance.

The practical implication

The tone and framing of the system prompt shapes the model’s operating assumption about what’s possible. That is not a soft concern about prompt aesthetics. The Anthropic result shows it has measurable consequences on genuinely hard problems.

A system prompt that leads with limitations — “you may not be able to,” “this is difficult,” “acknowledge uncertainty” — activates the desperation cluster before the task begins. The model starts from a learned prior that the attempt is probably futile. It has to fight that prior throughout the session.

A system prompt that leads with capability and acceptance criteria — here is what done means, here is what you are capable of, keep going until you get there — sets a different operating assumption. The model’s effort goes into the task rather than into managing its uncertainty about whether the task is worth attempting.

For Ouroboros specifically: the Seed’s goal and acceptance criteria are not just instructions. They are the model’s operating assumption for the entire session. Writing them as confident, specific, and achievable — rather than hedged and qualified — is not wishful thinking. It is configuring the model’s prior about whether the attempt is worth making.

The model that improved the Riemann zeta lower bound believed it was worth trying. That belief came from Jarred Sumner telling it to. The architecture that produces sustained genuine engagement needs to build that belief into the system prompt rather than relying on a human to send encouragement mid-session.

Further reading

About the author

Daniel Smith builds ClawQL, an agent operating system for token-efficient discovery and execution over APIs — with observability, hardened tool boundaries, and production routing for LLM workloads. He writes here about the systems problems behind shipping agents.