The Agent’s Operating System: why Harness Engineering is your next step to a reliable AI
Think of the model as the CPU and the harness as the Operating System. And here I show you how to do it on your PC.
The Agent’s Operating System: why Harness Engineering is your next step to a reliable AI
Think of the model as the CPU and the harness as the Operating System. And here I show you how to do it on your PC.
Most people spend all their time tweaking prompts (Context Engineering), but that’s only half the battle.
To build AI that actually works in the real world, you need Harness Engineering: the system of constraints, feedback loops, and orchestration that turns a raw model into a reliable worker.
Think of the model as the CPU and the harness as the Operating System.
Table of Contents
The "Perfect" Prompt That Failed
Defining the Discipline: Context vs. Harness
The Cockpit and the OS: Why "Harness"?
The Tools of the Trade: From Luxury Cars to Off-Roaders
It's Not Just for Coders: Delegating the Grunt Work
The Dark Side: The Context War and Token Burning
Where to Start
The “Perfect” Prompt That Failed
I remember the first time I thought I had “cracked” AI.
I spent weeks perfecting a RAG (Retrieval-Augmented Generation) pipeline. I tuned the retrieval parameters, I crafted the perfect system prompt, and I even added a few clever instructions to keep the model on track. It looked beautiful in my testing environment.
Then, I let it loose in a real-world scenario. I took few Engineering documents of my ongoing Oil&Gas plant to save me time browsing from one specification to the other and get the reply I need.
The results were frustrating.
The agent would work perfectly for three days, and then, on the fourth, it would hallucinate a tool call, ignore a critical piece of context, or simply loop in a useless way. It failed nearly 30% of the time. I was stuck in a cycle of “prompt engineering,” trying to fix every mistake by adding more words to the instruction.
I was optimizing the wrong layer. I was trying to fix a systemic failure with a linguistic patch. Truth to be told, generative AI tools for this job were not that accessible at that time.
But today, it is a complete different story!
Defining the Discipline: Context vs. Harness
This brings us to a distinction that I believe is the most important concept in AI engineering today: the difference between Context Engineering and Harness Engineering.
Most people use these terms interchangeably, but they are fundamentally different.
Context Engineering is what we have all been doing. It is the art of designing what the model sees at the moment of inference. It includes your system prompts, your RAG results, your tool definitions, and your message history. You rovide the right information so the model can solve the task. If context engineering is a success, the model gives a great answer right now.
Harness Engineering, however, is the discipline of designing the entire environment that surrounds the model. It is the “Operating System” that manages the agent’s behavior over time. It includes the constraints that prevent it from doing something dangerous, the feedback loops that catch errors, and the orchestration that manages complex, multi-step tasks.
If context engineering is about the content, harness engineering is about the control.
The Cockpit and the OS: Why “Harness”?
Why do we call it “harnessing”? The term isn’t just a fancy way to say “control.” It draws from two very powerful analogies.
1. The Cockpit (Human-Factors Engineering)
In 1947, researchers studied why pilots were crashing planes. They found it wasn’t “pilot error.” It was “cockpit error.” The controls were laid out in ways that didn’t match how humans actually behave under stress. If you redesign the cockpit (the interface), the pilot performs better.
In the AI world, the “Agent-Computer Interface” (ACI) is the cockpit. Harness engineering is the process of designing that interface so the model (the pilot) can fly the mission without crashing into a wall of hallucinations.
2. The Operating System
Large Language Model (LLM) can be compared to a high-performance CPU. A CPU is incredibly powerful, but on its alone, it can’t do anything useful. It can’t manage files, it can’t schedule tasks, and it certainly can’t talk to a network. It needs an Operating System to turn that raw computational power into a functional computer.
Harness engineering provides those essential “OS” services:
- Memory Management: Remembering what happened three steps ago.
- I/O Handling: Safely calling tools and interpreting their outputs.
- Scheduling: Deciding which sub-task to work on next.
- Error Handling: Knowing what to do when a tool returns an error.
The Tools of the Trade: From Luxury Cars to Off-Roaders
If you want to start building, you don’t have to start from scratch. We are seeing a massive convergence in the tools available, but they each have a distinct “personality.”
The Proprietary Heavyweights
If you want a polished, “it just works” experience, the big players have already built sophisticated harnesses.
- Claude Code (Anthropic): This feels like a luxury sedan. It is mature, deeply integrated, and has incredible “background routines” that can run tasks on a schedule. It’s powerful, but it’s a closed ecosystem.
- Gemini CLI (Google): This is the massive cargo ship. It is built around the idea of massive context windows (up to 2 million tokens). It is perfect when you need to shove an entire codebase into the agent’s brain all at once.
- Codex CLI (OpenAI): This is the high-performance racing machine designed for parallelism. It excels at “forking” sessions, allowing you to run multiple sub-agents in parallel to tackle different parts of a problem simultaneously.
The Open-Source and Agnostic Path
If you want to avoid vendor lock-in, or if you want to run everything locally for privacy (which I highly recommend!), there are other ways to go.
- OpenCode: This is my favorite “off-road vehicle.” It is model-agnostic. You can use the same agent definitions and the same skill files whether you are talking to GPT-4, Claude, or a local model running on Ollama. It’s about the protocol, not the provider.
If you don’t know from where to start, here is a good one
Not only for for Coders: Delegating the Grunt Work
One mistake I see people making is thinking that harness engineering is only for building coding assistants.
That is a massive missed opportunity.
The real power of a harness is its ability to handle “preparation jobs.” These are the repetitive, soul-crushing tasks that take up most of our professional lives. We can use agents to handle the grunt work, while we focus on the direction and the strategy.
The “Second Brain” and Knowledge Management
Imagine a personal intelligence system that acts as your librarian. Instead of you manually tagging notes and organizing folders, a harness-driven agent does the heavy lifting. It scrapes your web snippets, summarizes your meeting notes, and builds a structured knowledge base. You provide the high-level direction; the agent provides the organization. This is the essence of the “Second Brain” concept, but automated through a robust harness.
The “Knowledge Curator”
Think about how people manage massive amounts of information, like Andrej Karpathy’s LLM wiki. A harness-driven agent can be scheduled to monitor new research, summarize the key findings, and update a structured knowledge base. The agent does the reading and the organizing; the human does the synthesizing and the teaching.
I gave you the link to the first article of the series few lines above.
The goal is to move from being the “worker” to being the “director.”
The Dark Side: The Context War and Token Burning
However, I have to be honest: building a powerful harness comes with a heavy price. We are entering an era of the “Context War.”
In the old days of software engineering, we used “pipelines.” Data moved through a sequence of small, efficient steps. Each step only saw exactly what it needed. This was efficient, but it was also rigid.
In harness engineering, we are doing the opposite. We are trying to give the agent everything. We give it the whole file, the whole conversation history, the whole tool manual. We do this to increase the agent’s “intelligence” and reliability, but it creates a massive problem: Token Burning.
Every time a harness-driven agent makes a call, it often re-sends a huge amount of context. This leads to:
- Skyrocketing Costs: If you are using a frontier model like Claude 3.5 Opus, those extra tokens add up incredibly fast.
- Latency Issues: The more context the model has to read, the longer it takes to respond.
- The “Lost-in-the-Middle” Problem: Even the best models struggle when the context window gets too crowded. They tend to pay attention to the beginning and the end, but they “forget” the middle.
We are essentially fighting a war between capability and efficiency. A bigger harness makes a smarter agent, but it also makes a more expensive and slower one.
Where to Start
If you are feeling inspired to move beyond simple prompting, my advice is this: don’t try to build a massive system on day one.
Start with Context Engineering. Get your instruction files (like CLAUDE.md or AGENTS.md) solid. Teach the model your project's rules and conventions.
If you don’t have a clear project in mind try one of the following to see how the things are in action:
Second brain
Once that feels stable, move to Harness Engineering. Start adding simple constraints. Use a linter to check the agent’s output. Use a simple “ask-user” step for any action that involves deleting a file. Treat every failure as a signal to update your harness.
The goal is not to build a perfect agent. The goal is to build a system that learns from its own mistakes.
Leave me comments here below, and let me know how things worked out for you!
I hope you enjoyed the article. If this story provided value and you wish to show a little support, you could:
- Clap a lot of times for this story
- Highlight the parts more relevant to be remembered (it will be easier for you to find them later and for me to write better articles)
- Join my totally free weekly Substack newsletter here
- Follow me on Medium
- Follow my publication https://medium.com/artificial-intel-ligence-playground
If you want to read more, here are some ideas:
메타데이터
- post_id
- 211a89d887e7
- slug
- the-agents-operating-system-why-harness-engineering-is-your-next-step-to-a-reliable-ai-211a89d887e7
- url
- https://medium.com/artificial-intel-ligence-playground/the-agents-operating-system-why-harness-engineering-is-your-next-step-to-a-reliable-ai-211a89d887e7
- canonical_url
- https://medium.com/artificial-intel-ligence-playground/the-agents-operating-system-why-harness-engineering-is-your-next-step-to-a-reliable-ai-211a89d887e7
- author_url
- https://medium.com/@fabio.matricardi
- status
- ok
- fetched_at
- 2026-06-09 15:37:30
