90% Fewer Tokens, Same Answers — Here’s What Changed

All I Had to Do Was Stop Being Lazy About Context

Turns out your coding agent has been reading the entire phone book every time you ask it a question. Here’s the fix.

Okay so real talk — if you’ve been running Claude Code, Cursor, Codex, or literally any AI coding agent for more than a week, you’ve probably had that moment where you check your token usage and go “wait, HOW many tokens did that one bug fix cost me?”

If you are behind the Medium paywall and can’t read this article, click here this publication is open to everyone.

I had that moment last week. I asked my agent to fix one failing test. ONE. It read three log files, grep’d half my repo, and casually burned through 18k tokens before writing a single line of code. I stared at my screen like a man who just got a parking ticket for a car he doesn’t own.

So I went looking for a fix, and I landed on a tool called **Headroom, and honestly? It’s the kind of thing that makes you go “why didn’t I think of this” **while also being relieved that you didn’t have to build it yourself.

Headroom compresses everything your AI agent reads — tool outputs, logs, RAG chunks, files, conversation history — before any of it reaches the LLM. Same answers, fraction of the tokens. And they’re not shy about the numbers either — they’re claiming 60 to 95% reduction depending on the workload.

I tested it. I’m going to walk you through what it actually does, how to wire it up, and where it genuinely earned its keep.

Wait, what is Headroom actually doing?

Here’s the thing nobody tells you when you start using AI coding agents: most of your token spend isn’t your prompt. It’s everything the agent reads on your behalf — a git log, a stack trace, a 4,000-line JSON API response, your entire README.md because the agent decided it "might be relevant."

Headroom sits in between your agent and the LLM and basically asks: “do you really need all of this, or just the part that matters?” It runs locally (your data never leaves your machine, which I appreciated more than I expected to), and it routes whatever it sees — JSON, code, plain text, logs — into the compressor that’s actually built for that content type.

You can drop it in four different ways depending on how much you want to touch your existing setup:

• As a library — compress(messages) in Python or TypeScript, straight into your own code
• As a proxy — headroom proxy --port 8787, zero code changes, works with any language
• As an agent wrapper — headroom wrap claude (or codex, cursor, aider, copilot) in one command
• As an MCP server — exposes headroom_compress, headroom_retrieve, headroom_stats to any MCP client

I went with the wrap option first because I am, fundamentally, a person who values not editing fourteen config files on a Tuesday.

Setting it up (genuinely took me under a minute)

# Install — pick your poison
pip install "headroom-ai[all]"          # Python
npm install headroom-ai                 # Node / TypeScript

# Wrap your agent - no code changes, no drama
headroom wrap claude

# Check what you saved
headroom perf
headroom dashboard   # live savings dashboard, proxy must be running

That’s it. That’s the whole setup. I kept waiting for the part where I had to edit an .env file three times and restart my terminal in a specific order, and it just... didn't come.

The part where I stopped being skeptical: the actual numbers

I’m naturally suspicious of any tool that says “up to 95% savings” because “up to” is doing the same heavy lifting as “results may vary” on a weight loss ad. So I want to give you their published numbers as-is, because they’re more specific than I expected:

And on the demo GIF on their repo, a live run goes from 10,144 tokens down to 1,260 — same fatal error found, just without dragging the entire log file along for the ride.

What got me more than the compression numbers, honestly, was the accuracy table. Because compression that breaks your answers is just a worse, cheaper way of being wrong. So it’s not just throwing your context in a blender. It’s actually keeping the parts that matter.

How it works under the hood (without the PhD)

Here’s the pipeline, roughly:

Your agent / app
  (Claude Code, Cursor, Codex, your own code…)
       │  prompts · tool outputs · logs · RAG results · files
       ▼
   Headroom (runs locally)
   ├─ CacheAligner    → stabilizes prefixes so KV caches actually hit
   ├─ ContentRouter   → detects content type, picks the right compressor
   │    ├─ SmartCrusher    (JSON)
   │    ├─ CodeCompressor  (AST-aware)
   │    └─ Kompress-base   (prose, their own HuggingFace model)
   └─ CCR             → caches originals locally for retrieval
       │  compressed prompt + retrieval tool
       ▼
LLM provider (Anthropic · OpenAI · Bedrock · …)

A few pieces here genuinely impressed me:

SmartCrusher handles your JSON blobs — arrays of dicts, nested objects, the stuff your API responses are made of. If you’ve ever pasted a 200-line API response into a chat just so the model could read the three fields you actually cared about, this is the thing that stops you from doing that.

CodeCompressor is AST-aware, meaning it understands Python, JS, Go, Rust, Java, and C++ as actual code structures, not just blocks of text it can chop arbitrarily. That matters a lot if you’ve ever had a naive truncation tool cut a function off mid-loop and confuse the model into “fixing” code that was never broken.

CCR (reversible compression) is the one I trust the design philosophy of the most. Originals get cached locally, and if the LLM decides it actually needs the full version of something, it can call headroom_retrieve and get it back. So you're not losing data — you're deferring it until it's actually needed.

from headroom import compress

# Real-world scenario: your agent just pulled a giant
# API response while debugging a flaky integration test
raw_messages = [
    {"role": "user", "content": "Why is the /orders endpoint timing out?"},
    {"role": "tool", "content": huge_json_api_log},  # 12k tokens of noise
]
compressed = compress(raw_messages, model="claude-sonnet-4-6")
# compressed now carries the signal - status codes, error fields,
# timing anomalies - without every successful 200 response
# riding along for no reason

The thing people sleep on: output tokens

This is the part I didn’t expect to care about, but now genuinely do. Everyone talks about shrinking what you send to the model. Almost nobody talks about what the model writes back — and on Opus-class models, output tokens cost five times as much as input tokens.

A lot of that output is pure ceremony. “Great, let me look into that for you,” followed by the model re-printing code you already showed it, followed by three paragraphs of “thinking” about a file read that didn’t need thinking about. Headroom has a flag for this:

export HEADROOM_OUTPUT_SHAPER=1     # off by default
headroom proxy --port 8787

It appends a short “be terse, don’t restate context” instruction to your system prompt (placed so your prompt cache still hits — small detail, big deal if you care about cache costs), and it dials down “thinking effort” on turns that are just the model resuming after a routine tool result, like a passing test or a successful file read.

And because they can’t actually see what the model would have written without the shaping, they report it honestly as an estimate with a confidence range instead of a suspiciously clean percentage:

headroom output-savings
# Reduction: 31.7%  (95% CI 27.7% … 35.7%)   [estimated]

If you want a measured number instead of an estimated one, you can hold out a slice of conversations as a control group:

export HEADROOM_OUTPUT_HOLDOUT=0.1

I respect a tool that admits when it’s estimating instead of just printing a big shiny number and hoping you don’t ask follow-up questions.

headroom learn — the part that feels a little like magic

This one’s a sleeper feature. headroom learn mines your failed agent sessions — the ones where the agent got something wrong or you had to correct it — and writes the corrections straight into your CLAUDE.md, AGENTS.md, or GEMINI.md. So instead of you manually updating your instructions file every time the agent does something dumb, it learns from its own mistakes and writes the fix down for next time.

There’s a sibling feature for verbosity specifically — because nobody actually tells an agent how terse they want it, they just show it by interrupting long replies or moving on before reading them:

headroom learn --verbosity            # preview what it found, dry run
headroom learn --verbosity --apply    # commit to it

Quick wins worth knowing about

Agent wrapping across your whole stack — headroom wrap supports Claude Code, Codex, Cursor, Aider, Copilot CLI, OpenClaw, and Cortex Code, and Codex shares memory with Claude. Use this if you bounce between agents during the day and don't want each one starting from a blank slate.

Cross-agent shared memory — SharedContext().put / .get lets multiple agents share compressed context with provenance tracking and auto-dedup. Reach for this if you're running a multi-agent setup and tired of re-explaining the same codebase to three different tools.

Framework middleware, not just a CLI — there’s withHeadroom(new Anthropic()), a Vercel AI SDK middleware, a LiteLLM callback, a LangChain wrapper, an Agno model wrapper, and an ASGI CompressionMiddleware. Reach for these if you're building your own agent on top of an SDK rather than using an off-the-shelf coding tool.

# Example: dropping it straight into the Anthropic SDK
from anthropic import Anthropic
from headroom.integrations.anthropic import withHeadroom

client = withHeadroom(Anthropic())
# every call through this client now gets compressed automatically

My honest take

I went in expecting another “just truncate the middle of your context” tool with a fancier README. What I got instead was a router that actually understands the shape of what it’s compressing — JSON stays structured, code stays AST-valid, prose gets summarized by an actual trained model — plus a reversible cache so you’re never really losing anything, just deferring it.

The output-token-shaping piece is the one I didn’t see coming and now can’t stop thinking about, mostly because I’ve definitely paid for a paragraph of “Let me think through this carefully…” before a one-line fix more times than I’d like to admit.

Suggestions and Recommendations

• Start with headroom wrap before the library mode. Zero code changes, and you'll see your real savings number before deciding if it's worth deeper integration.
• Turn on HEADROOM_OUTPUT_SHAPER=1 early. Input compression is the headline, but output savings compound fast on Opus-class models where output costs 5x input.
• Use headroom learn --verbosity after a week of normal use, not on day one — it needs real sessions to learn your actual terseness preference, not your stated one.
• If you’re on a corporate network with SSL inspection, install Rust first before pip install — the build backend fetches rustup over a connection your TLS stack might reject.
• Check headroom dashboard after your first few sessions instead of trusting the README numbers blindly — your workload's compression ratio will differ from code search or SRE debugging.
• If you run multiple agents (Claude + Codex, say), enable the shared memory mode before you end up explaining your repo structure to each one separately, again.

Key Learnings

• Most of your token spend isn’t your prompt — it’s what the agent reads on your behalf. Logs, JSON blobs, and stale file contents are the real budget killers.
• Compression that breaks accuracy is just an expensive way to be wrong. The benchmark results matter more than the compression percentage alone.
• Output tokens are the quiet half of your bill. Nobody optimizes them because nobody’s watching them, and on Opus-class pricing that’s a 5x blind spot.
• Reversibility changes the calculus on aggressive compression. If the model can always ask for the original back via CCR, you can compress harder without fear.
• AST-aware code compression beats naive truncation. A tool that understands a function boundary won’t accidentally amputate a loop mid-body.
• Agent wrapping is the lowest-friction entry point. If a tool needs you to rewire your whole stack before you see a benefit, that’s a red flag — headroom wrap is the opposite of that.

Have you tried trimming your agent’s context before, or are you still letting it read your entire repo for a one-line fix like the rest of us? Let me know in the comments.

Follow for more test automation and AI tooling deep-dives.