Trust and Scepticism (3): Designing Agents

In this third post about my wargaming and scenario-analysis tool I can finally write about one of the most exciting aspects of the work for me: designing the agentic framework. I’ll structure it as follows — first I’ll go over my inspirations and beginnings, then some of the learnings and rules I established, and lastly a high-level overview of the different roles and their connection to wargaming.

On Gastown and New Atlantis

When I first learned about Gastown by Steve Yegge (https://steve-yegge.medium.com/), I read through his articles on Medium and visited his GitHub. Naturally, I also came across his multiple warnings about the complexity and the steep learning curve. I chose to ignore it somewhat. Of course I wasn’t ready. I think I’m still not fully there, yet. But I was motivated, and dedicated more of my free time than was probably sensible to diving into the thick of it. Beads to assign and track workloads, orchestrators to align worker agents — I learned a lot here.

New Atlantis, as a philosophical offshoot of Gastown (https://github.com/shaharavin/new-atlantis/tree/new-atlantis-mvp/first-works), gave me the spark to kickstart my own development. Shahar and his colleagues adapted Gastown to build philosophical agents that debate problems between each other and provide detailed analysis to the operator. I studied both repos and the accompanying articles to see what I could adopt. Initially I wanted to build something similar for understanding philosophical challenges from different angles.

But, as described in my first post, I changed my mind and pivoted towards a wargaming and analysis platform. I co-lead some industry working groups for military AI and actively participate in a few NATO Research Task Groups (RTG). The use of LLMs for training, decision-support, image analysis and other tasks is one of the focus areas there. That only adds to my motivation for this project, as it helps me contribute even more.

Basic Hygiene

These are some decisions I made quite early on, once I’d identified them as necessary preconditions for a tool meant to be used in the military or security domains.

First, every model call goes through an abstraction layer, in my case LiteLLM, so there’s no hard-coded vendor lock-in. Which model an agent uses is a matter of configuration, not code, the same orchestration layer can point at a hosted API or at a local endpoint (vLLM, Ollama, and the like). On-prem self-hosting isn’t a finished, benchmarked deployment yet, but the plumbing and the setup documentation are in place as a standard step, so moving a framework into an air-gapped environment is a configuration issue, not a rewrite.

Second, I wanted my agents to run on doctrine and other authoritative documents for the respective use case. To enforce that separation, each framework has its own doctrine table, so targeting manuals can’t drown out route-planning documents, alongside a couple of shared tables (general and wargaming material) that every framework is allowed to draw on. Retrieval is then constrained per agent: each agent declares a budget of sources and chunks, so a large document can’t dominate a smaller but more authoritative one simply by virtue of its size.

Third, agents are required to label the provenance of what they produce, [DOCTRINE], [TRAINING], or [ASSUMPTION],so the operator can see at a glance whether a statement is grounded in an ingested doctrine, drawn from the model's training data, or an inference the agent is making on its own. It's an enforced authoring contract, checked in the test suite, not a polite request. I'll go deeper on this in a later post; for now the point is that this grounding is meant to surface uncertainty rather than hide it, and to make fabrication easier to spot.

Provenance Labelling in action

Agentic Baby Steps

I started with a single framework: the joint targeting cycle. It involves ten role-specific agents: an intelligence officer, a targeteer, a legal advisor, a collateral-damage analyst, a weaponeer, and so on , wired together through three small reusable protocols and an orchestrator that walked them through the phases of the doctrine. I’d absorbed enough from Gastown and New Atlantis to start at “many agents arguing with each other” rather than “one clever prompt I’ll split up later.”

A few decisions from those first two weeks turned out to be load-bearing, and I’ve never regretted them.

The first was to let the doctrine set the structure, not me. AJP-3.9 (Joint Targeting Doctrine) defines the phases, the roles, and the order they act in; the agent roster is a fairly direct mapping of that document. Whenever I was unsure whether some role belonged in the system, the primary question was “does the doctrine name it?” — not “would it be neat to have?” That is something I want to consider later on, but for now, supporting the flow instead of re-designing it is the task.

The second was to stop hoping the model would produce structured output and start requiring it. Early on the agents wrote their assessments as free text with section headings, and I read them afterwards. While convincing, that was unreliable in many ways. The fix was to make every agent submit its decision through a typed tool call, a schema with the fields spelled out and the GO/NO-GO choices as hard enums, so a gate can’t hand back a verdict the code can’t read. Getting a binding legal or collateral-damage decision out of a language model, reliably, every single time, was the first genuinely hard problem I hit. And from there, I adapted it to various other frameworks and necessary gates.

Hard Gates and how to deal with them

The third was to put the gates in code, not in the prompt. A legal or collateral-damage STOP doesn’t depend on the model being in a cooperative mood, it’s enforced by the control flow, and a human can only step past it through a logged, accountable pause. That principle was there from the start, and everything since has only hardened it.

The bigger structural change came about a week in. What began as “the targeting app” clearly had potential for a whole platform: the same machinery, agents, protocols, doctrine tables, an orchestrator, a pause for meaningful human control, was going to be needed for tactical planning, for fires, for route planning, etc. So I demoted the targeting cycle from “the system” to “the first framework,” and pulled everything shared into a common core that new frameworks plug into. That refactor is why there are several frameworks today instead of one fork per use case and why a fix to the shared machinery now reaches all of them at once.

The last piece to fall into place — and the most recent — was the layer of agents that don’t do the work but judge it. That idea didn’t arrive fully formed either. It was prototyped inside the scenario-analysis framework first, as a single critique agent, before I realised it was general enough to belong to every framework and started lifting it out into the shared core. I’ll talk more about that in the following section.

The Four Layers

Conceptually, each framework is built around Four Layers of Agents.

The archteypical agent layers

The Operational Layer holds the roles you’d expect for the use case — for military decision-making, the staff: intelligence, logistics, fires, and so on. They’re modelled on their real counterparts and do the work the scenario, the ingested doctrine, and their system prompt assign them. They pass tasks and information to each other, produce output, and hand it on. Sometimes they disagree — over the identification of a target, or the assignment of forces — and that disagreement runs in loops until they reach something everyone can live with, or the commander (the operator) is asked to make the call and move the process along. Each Operational Layer is built from scratch according to the task.

The Wargaming Layer contains all the agents and artifacts which are needed to turn a framework from a mere procedural training session into an interactive simulation. It introduces enemy reactions to the operators decisions, introduces uncertainties and random developments and analyses taken actions. This layer is informed by the various wargaming manuals out there and confronts the agents or the operator with the possible consequences of their actions.

The Red-team Layer sits above them, and needs some explanation. Red-teaming here does not mean playing the enemy, the simulated adversary lives down in the operational layer with everyone else. I put them there because they also do scenario-work. The red-team layer is an adversarial reviewer. It attacks the staff’s own output: where are the brittle assumptions? What failure mode got ignored? Why the GBU-12 and not the GBU-38 for this target? Its job is to make the workers defend their choices and to shake loose the biases that creep in when a role is confident. It is here where I also advocate to use models different from the workers, in order to elicit better reviews and defenses by preventing same-model-sycophancy.

The Meta-Critique Layer sits above that, and it’s a different kind of scrutiny again. It doesn’t attack the decision and it doesn’t care whether the answer is “right.” It evaluates how well the work was reasoned — was the method actually followed, is the provenance labelling honest, did the agents lean on training-data guesses where doctrine was required? Red-teaming asks “how does this fail?”; meta-critique asks “how sound was the thinking that produced it?”

Adding in a sprinkle of philosophy, maybe you could interpret the red-teaming layer, who are looking at the consequences, as being utilitarian. And maybe the meta-critique team follows a deontological approach. I will need to formulate that out in another article I think.

Roles and Friction on the Seams

The decision cycle is modelled via distinct doctrinal roles — a targeteer, a legal advisor, a collateral-damage estimator, a fires coordinator — each one an agent, each moving through the phases of the doctrine. The commander’s chair is deliberately not an agent: the GO/NO-GO decision stays with the human operator, advised by a set of strategist personas but never made by them. The most consequential seat in the process is the one I kept human on purpose. But, quite easily, I could have the whole cycle run autonomously and provide the AI-generated targeting solution or order of battle. This could be used to compare human solutions, for example. Just as easily, the targeteer could be replaced by a human, or the logistics officer. It is all about modularity. The training possibilities are quite numerous.

Why did I follow the doctrinal role-process so religiously? Why did I not just create a monolithic targeting cycle agent doing it all? Three reasons, and the third is the important one:

• It mirrors the real staff process the operator has to learn. There is even a S3NCO agent, the oversight_monitor, running around in the background, whipping the other agents if they fail to do a correct hand over, good times.
• It makes disagreement visible — the legal advisor and the targeteer can openly conflict, the way they do on a real staff.
• A system of separate roles has seams. A single monolithic answer has nowhere to slip a plausible-but-wrong input and see whether the trainee notices. Roles give you those seams — and the seams are exactly where the Critical Trust Environment from the first post does its work.

Where a gate stops the cycle, it doesn’t just say no. It names what’s missing and from whom, and consults that role before anyone retries — no blind loop of the same failure. A human can override a gate — but the override is recorded, attributed to the operator, and captured with their stated reason, so overruling the AI always leaves a trail. The point is that distrust of the AI’s judgment is written into the control flow, not left to the operator’s discipline.

Developing Core Roles

Sometime along the way, I split my work into three distinct areas. The first is scenario analysis. I worked in this field mostly during my studies, using techniques like the intuitive-logics approach to develop and analyse scenarios around the security of semiconductor supply chains. The second area is wargaming. I’m a huge wargaming fan, both privately — as a decade-long Warhammer player — and professionally, where I attend conferences and training whenever I can. The third is general decision support, which is very much still under development and draws on elements from the other two. I’ll cover each of these in more detail in later posts.

That split gave me an idea. By now these three areas have grown into a whole family of frameworks — enough that keeping a separate copy of every supporting role inside each one already is a real maintenance burden. So I took a step back and asked what could be centralised. The worker agents were ruled out quickly: they’re framework-specific and rely on context that doesn’t generalise. There might be an argument for a role like the military S3 officer to be universal, but its job in a targeting cycle is different from its job in a “normal” decision-making process. I focussed on clearer cases, but still developed a common agent writing guide along the way that is being used for every new agent I create.

Workers stay individual, attached to the phases of their own framework. But the layers above them — wargaming agents, the red-team and meta-critique roles — are abstract enough to generalise. They now live as a single shared contract in the core, rather than as a bespoke copy inside every framework, and each one is handed (1) the agents, interactions and content it needs to critique or interact with, and (2) the context it’s doing that in. The extraction itself is done; wiring each framework to actually call the shared version is a staged rollout still going in framework by framework. These things take time I learned.

Lessons learned in designing Agents

A wrong turn worth admitting. Early on, the legal-advisor agent kept getting cut off mid-answer. I spent days treating it as a technical bug and restructured the machinery half a dozen times over two days. The cause was a wrong diagnosis. The real problem was that the agent was writing 500-word legal essays and hitting a length ceiling. The fix wasn’t technical at all — it came from a domain question: should a legal advisor making an operational decision write a 500-word essay, or a short fielded assessment a commander can act on in seconds? Quite obviously the latter. The first LEGAS agent was trying to argue his case in Den Haag instead of providing concise field assessments.

Once every agent was told to answer like a staff officer under time pressure instead of a law review for a journal, the problem evaporated. The lesson I keep from it: in this domain, the right answer is more often doctrinal than technical.

Where it’s heading

The system didn’t arrive fully formed — it walked backwards into its own thesis, and I’ve traced most of that walk above: doctrine-first, structured output, gates in code, then the pivot to a platform and the layers that judge the work.

How the current system evolved

Rather than replay it, here’s the part I haven’t shown yet.

Hybrid intelligence — in progress. The newest layer grounds the wargaming in doctrine-bound stochastic models: Lanchester-style attrition, a Markov model, is wired into the wargaming step of one framework today. Reinforcement learning and cross-framework transfer are the next stretch, architecture-decided and prototyped in research spikes, but not yet shipped. The shared wargaming artefacts themselves, decision-support matrices, synchronisation and risk worksheets, already run across several frameworks.

Every one of these turns came from live operator feedback or from research, RAND’s work on AI in wargaming, hands-on spikes with multi-agent RL toolkits like PettingZoo, or other sources. And they all serve one premise: operators must remain skeptical towards the AI. Not because the AI is untrustworthy, but because these decisions are too high-stakes to fully automate. The architecture’s job is to make that distrust observable and structural.

Thank you for reading! In my next post I will write about scenario analysis as one of my central framework classes. It is pretty well advanced and I am quite happy with the quality of the content yet, but of course there are still some challenges to overcome.

If you have questions, tips or criticism, I would love to connect.

Best,

Matthias