Both Sides: Why Input Compression Alone Isn't Enough

The MCP token efficiency conversation has converged on one insight: tool catalogs are expensive. Send 44 schemas per request and you pay for 44 schemas per request. Solutions range from CLI approaches that eliminate schemas entirely to on-demand fetching that loads schemas only when the model asks for them.

Every one of these solutions addresses the input side. None of them addresses what the model sends back.

This is a structural gap. An API response that returns 50 fields when you needed 3 adds those 47 fields to the model’s context on the next turn — where they become context bloat, the largest single cost driver in most agentic workloads. Fixing input inflation while ignoring output inflation solves half the problem and leaves the other half compounding across every subsequent turn.

ClawQL’s search → execute design solves both sides through the same two primitives. search eliminates input inflation. execute uses GraphQL projection to trim the output before it enters the context window. The combination produces three compounding effects: lower token cost, better model quality, and higher cache hit rates. Each has published research behind it. This post connects the design to the evidence.

This pairs with The Twelve Layers of LLM Cost, The API Spend That Never Compounds, Seven Surfaces, One Catalog, and the agent memory stack. For measured numbers against a codemode gateway on the same task, see Both Sides of Context Compression.

The input side: how search handles it

The input problem is well-understood. A GitHub API spec is roughly 2.5 MB of JSON — approximately 625,000 tokens. Registering it as MCP tools means the model sees some fraction of those schemas on every turn. Even with on-demand fetching or lazy loading, the tool discovery overhead exists.

search takes a natural-language intent and returns the specific operation IDs and parameter hints the agent needs for that step. The spec stays server-side. The model sees 200 tokens instead of 625,000. This is the same principle as the CLI approach that Alier et al.’s August 2026 paper identifies as its best performer — the model uses knowledge from pretraining rather than consuming a schema description on every request.

What’s different is that search works across any API surface: REST, GraphQL, gRPC, WebSocket, CLI, or native MCP servers. The same two primitives handle everything. You don’t swap interfaces when the underlying service changes.

The output side: what execute adds

When an agent calls a REST endpoint, it gets back what the API sends. A /users/{id} response might include 40 fields. The agent needed name and email. The other 38 fields enter the context window as tool output, where they begin accumulating in subsequent turns.

execute passes a GraphQL projection alongside the operation. The agent specifies which fields it wants. The response is shaped to those fields before it leaves the server. 38 unnecessary fields never enter the context window.

This is not a minor optimization. MindStudio’s April 2026 analysis found that filtering a 50-field response to 3–5 fields reduces payload tokens by 80–90%. Tool outputs are a primary source of context accumulation in agentic sessions. Trimming them at the source rather than at the application layer is the structural fix.

The projection syntax is straightforward:

// Without projection — full response enters context
await execute('github.issues.get', {
  owner: 'org',
  repo: 'repo',
  issue_number: 42,
});
// Returns: 40 fields including timeline_url, node_id, author_association,
// active_lock_reason, performed_via_github_app, state_reason...

// With projection — only requested fields enter context
await execute(
  'github.issues.get',
  {
    owner: 'org',
    repo: 'repo',
    issue_number: 42,
  },
  {
    fields: ['number', 'title', 'body', 'state', 'assignees.login'],
  }
);
// Returns: 5 fields

The agent writes this projection as part of the execute call. The compression happens at the gateway layer before the response reaches the model.

Why both sides matter: context size and quality

Reducing token count is usually framed as a cost argument. The research says it’s also a quality argument, and the quality effect is larger than most practitioners expect.

Context rot is measurable and consistent across models. Chroma Research’s 2025 study of 18 frontier models found performance degradation at every increment of context growth, across every model tested. They named this phenomenon “context rot.” It doesn’t produce errors. It produces subtly wrong outputs that pass basic validation and fail when a human reads them carefully.

The NoLiMa finding is the strongest evidence. The NoLiMa benchmark (Modarressi et al., ICML 2025, LMU Munich and Adobe Research) is the most rigorous measurement of this effect. The benchmark removes lexical shortcuts — questions and needle passages have minimal word overlap, forcing genuine reasoning rather than keyword matching. Results across 13 models claiming 128K+ context support:

At 32K tokens, 11 of 13 models dropped below 50% of their short-context baseline scores
GPT-4o fell from 99.3% at short context to 69.7% at 32K
GPT-4.1, despite claiming 1M context, had an effective context length around 16K before performance degraded significantly

The paper’s explanation: transformer attention mechanisms have increasing difficulty resolving indirect associations in longer contexts. Irrelevant tokens in the middle of a long context actively compete with relevant tokens for the model’s attention. Adding more content doesn’t add more capacity. It dilutes it.

Stanford’s “Lost in the Middle” finding (Liu et al., 2023) established the same mechanism: information at the beginning and end of context is attended to most strongly. Information buried in the middle of a long context is routinely missed. This is not a model bug. It’s how attention works at scale.

The JetBrains research team’s 2025 study (Efficient Context Management) compared context management strategies across 250-turn SWE-bench trajectories and found something counterintuitive: simple observation masking — hiding irrelevant earlier content — improved solve rates by 2.6% while cutting costs by 52% compared to no context management. Less context, better outcomes.

The implication for API tool use is direct: tool outputs that aren’t needed for the current step should not be in context for the current step. GraphQL projection enforces this structurally, at the infrastructure layer, on every call.

The third effect: cache hit rates

Prompt caching reduces input token cost by 90% (Anthropic) or 50–90% (OpenAI, Google) for repeated context prefixes. The economics are significant. The problem most teams encounter is that their cache hit rates are far lower than expected.

The reason is almost always the same: dynamic content is positioned too early in the prompt, causing it to invalidate the stable prefix on every turn.

ProjectDiscovery documented this precisely. Moving a single dynamic identifier from the middle of a prompt to the end took their cache hit rate from 7% to 84%, cutting their monthly inference bill by 59%.

ClawQL’s architecture produces this structure by design. The system prompt and tool surface are minimal and static — just search and execute with their schema descriptions. The dynamic content — tool results, conversation turns, agent reasoning — is always appended at the tail. The stable prefix is maximally large as a fraction of total context.

When both sides are compressed:

Input tokens per turn are smaller (search returns operation hints, not full schemas)
Output tokens per turn are smaller (execute returns projected fields, not full responses)
The entire context grows more slowly across a session
The stable prefix remains a larger fraction of total context for more turns
Cache hit rates stay higher for longer

The cache benefit compounds. A turn-20 session where context has been trimmed on both sides costs dramatically less than a turn-20 session where tool outputs have been accumulating unchecked — and the compressed session produces better outputs because the model’s attention isn’t spread across 19 turns of accumulated field noise.

What this looks like in numbers

The twelve layers post documents a before/after for a document processing workload at 10,000 calls per day. The relevant rows:

Layer	Before	After	Daily saving
Context bloat (input)	8,000 tokens/call	800 tokens/call	72M tokens → $216
Verbose output	1,200 tokens/call	400 tokens/call	8M tokens → $40
Redundant static (cache)	2,000 tokens uncached	200 tokens cached	18M tokens → $54

These three rows don’t add independently. Fixing the output side (Layer 2) reduces context accumulation across turns, which magnifies the cache benefit (Layer 3), which reduces the effective input token count (Layer 1) further. The compounding is real but doesn’t show up when you analyze each layer in isolation.

The 862× compression figure in ClawQL’s context compression benchmark — 7,174 endpoints indexed, 62 operations surfaced, context dropping from ~10.2M tokens to ~12K across a three-provider workflow — reflects both sides operating simultaneously. Eliminating input schemas accounts for most of it. GraphQL projection on the outputs accounts for the rest. Neither alone produces that number.

Why nobody else does both sides

The input-side approaches are well-established:

CLI tools (pi, Tau from the Alier et al. paper) eliminate tool schemas entirely by using the model’s pretraining knowledge of shell commands
On-demand schema fetching (Hermes’s approach) reduces schemas per request from 44 to 7
MCP code execution (Anthropic’s engineering approach) defers tool descriptions until the model needs them

Each of these treats output as a separate concern — or doesn’t treat it at all. The agent gets the full API response and the application layer handles trimming.

The output-side approaches also exist:

Apollo MCP Server surfaces GraphQL operations as MCP tools, letting agents write queries that return only requested fields
MindStudio recommends field filtering at the MCP server layer as a manual server-side concern
Various field selection middleware can be applied before responses reach the model

Each of these treats input as a separate concern — or relies on the agent already knowing which operation to call.

ClawQL’s search → execute is the only unified primitive that handles both in a single two-step flow: search reduces what goes into the request, execute with GraphQL projection reduces what comes back. The compression on both sides happens through the same two tools that agents already use for everything else. There’s no separate field-filtering layer to configure, no separate CLI adapter to maintain alongside the MCP surface. It’s one interface across REST, GraphQL, gRPC, CLI, and native MCP servers simultaneously.

The practical argument

If you’re running agentic workflows over APIs, the token economics work out like this:

Without output compression, tool results accumulate. By turn 10 of an agent session, the model is reasoning through the outputs of 10 prior tool calls plus whatever it retrieved from each one. If each response returned 50 fields and the agent needed 5, that’s 450 unnecessary fields sitting in context, competing with the relevant content for attention, inflating the prefix on every subsequent turn, and degrading the cache hit rate as the session grows.

With output compression, each tool call adds only the fields that matter. Context accumulates at the rate of signal, not at the rate of API verbosity. The cache stays effective longer. The model reasons over less noise. The outputs are better.

The cost and quality arguments point in the same direction. Smaller, more precise contexts cost less per token, produce better outputs per token, and keep more tokens in the cache rather than billing at full price. The research — NoLiMa, Chroma’s context rot study, Stanford’s lost-in-the-middle finding, JetBrains’ observation masking results — consistently supports this direction.

Solving only the input side is solving half the problem.