The Last Argument Always Wins

So I built a council.md. A file-first LLM council to make models disagree before they persuade me.

I’ve been thinking about this Karpathy tweet for a while and it makes me think about something I’ve learned from Chris Williamson.

Thanks to evolution, our brain has a stronger horsepower engine to see the glass as half empty, Charlie Munger found a way to harness this with his famous quote:

invert, always invert.

LLMs are dangerous because they are good at making whichever side you asked for feel reasonable.

I noticed something annoying about myself. When a model argued well, I could feel my opinion moving. Not because the model was right. Because the argument was clean.

They can invert almost anything.

The last answer has too much power

That makes the user part of the system more important than we like to admit. If I ask a lazy question, I get a persuasive lazy answer.

If I ask for confidence, I often get something that sounds like confidence.

And if I ask the opposite, I may suddenly believe that too.

Most AI councils agree too beautifully

The obvious answer is to use multiple models.

Get a strategist. A critic. A user advocate. A risk analyst. Let them argue. Then synthesize.

This sounds rigorous. Sometimes it is.

But a lot of AI councils have a quiet failure mode: they agree too early.

One agent frames the problem. The others react to that frame. The discussion starts leaning. Then the synthesizer arrives and turns that lean into a neat conclusion.

Now the user is not looking at one chatbot answer. They are looking at “the council.”

That sounds stronger.

It may not be.

A bad council does not fail by arguing. It fails by agreeing too beautifully.

The real problem is who saw what

This is the part I kept coming back to.

I believe we can reach useful disagreement with a few basic questions like:

• Who saw the original context?
• Who saw another agent’s answer?
• Did the critic write independently, or did they react to the builder’s framing?
• Did the synthesizer preserve the minority view?
• Did the human decide, or did the summary gently decide for them?

That is why I built *council.md*

It is a protocol for running AI councils using plain session files. The basic flow is simple:

1. A coordinator helps define the question.
2. Agents write independent first drafts, ideally blind.
3. Their contributions are merged into a shared discussion.
4. A synthesizer maps agreement, disagreement, open questions, and candidate paths.
5. The human makes the decision.

The important part is that the information flow is visible.

For example, if the Builder says

“this is mostly a UX problem”

before the Critic writes, the Critic may spend its whole turn reacting to UX. But if the Critic writes blind, it may decide the real issue is distribution, pricing, or trust. That difference matters.

I’m not against runtimes. I’m against hidden reasoning.

The first version of this idea was easy to describe as “an LLM council without a runtime.”

I still like that phrase. It has a nice bluntness to it.

But it is not quite the mature position.

The better position is:

file-first, invariant-first, runtime-optional.

A runtime is fine if it preserves the protocol. If it exports the session files, keeps blind rounds isolated, logs who saw what, and leaves the human in control of phase transitions, great.

What I do not want is hidden orchestration.

I do not want a system where five agents supposedly deliberated, but I cannot reconstruct the prompts, the order, the memory, the handoffs, or the moment synthesis began.

I’m not religious about files.

I’m religious about being able to reconstruct the argument.

Blind first, argue later

The blind first round is probably the most important design choice.

Each agent gets the same context and its own role. It writes its first position before reading the other agents.

It means the risk analyst is not anchored by the strategist. The user advocate is not politely reacting to the builder. The challenger has to actually challenge from its own lane, not just decorate someone else’s framing.

After that, you can run a second round. Let them respond. Let them clash.

But the first thoughts are preserved.

That matters because once a strong framing enters the room, everything after it starts bending around that framing.

Files are boring in exactly the right way

council.md is file-first.

A session is just a folder:

This is not because markdown is magical.

It is because files are inspectable. They make the process harder to hide.

You can open the folder and see the actual record:

• this was the question
• these were the roles
• this is what each agent saw
• this is what each agent said
• this is where synthesis happened
• this is where the human decision was recorded

The files are not the interface.

The files are the audit trail.

The optional briefing is generated from the synthesis. It is not the decision.

The synthesizer is not a judge

The synthesizer has one job: map the territory.

It should not turn disagreement into a soft consensus because that reads better.

A good synthesis should say:

• here is where agents independently converged
• here is where they disagree
• here is what is at stake in that disagreement
• here is what evidence would resolve it
• here are the candidate paths
• here is what remains unknown

The synthesizer is not there to make me feel done.

It is there to make me harder to fool.

The useful part is often the conflict that remains unresolved.

The UI is a briefing, not the source of truth

I added an optional UI because reading raw markdown is not always pleasant.

The UI turns the synthesis into a staged briefing: overview, agents, agreements, conflicts, risks, candidate paths, open questions, and a final “before you decide” moment.

But the UI is deliberately non-authoritative.

It is generated from JSON inside synthesizer.md. The model fills the JSON. The CLI renders the HTML. The HTML does not become the source of truth.

If the UI ever disagrees with the markdown, the markdown wins.

The final step is still human judgment.

Subscription-native is not a compromise

One thing I care about more than I expected: this should work with the AI access people already have.

Most people do not have a clean API setup across every model. They have subscriptions. Browser tabs. Claude. ChatGPT. Gemini. Perplexity. Grok. Maybe a local model. Maybe Cursor or Claude Code.

So council.md treats copy-paste as a first-class workflow.

That is slower than a runtime. It is more manual. It can be annoying.

But it also means the protocol works without API keys, billing setup, provider lock-in, or a custom orchestration service.

For occasional hard decisions, that tradeoff feels right.

The point is not to automate everything.

The point is to make the thinking process visible enough that I can trust the record.

The council now reviews itself

There is a funny consequence of building a protocol like this.

At some point, the repo itself has to live by the rules.

So council.md now has a self-improvement profile. Meaningful changes to the project are supposed to go through a council before becoming a PR.

In fact the first version of this idea, the “an LLM council without a runtime.” did not survive the council!

So I scrapped it.

That sounds a little ridiculous.

I kind of like that.

Not because I think the project is now some self-improving AI organism. It is not. There is still a human deciding. There is still a maintainer judging the change.

But if the whole thesis is that meaningful changes should survive structured disagreement, then the protocol should be able to survive its own process.

Especially when the change is about the protocol itself.

Where this might be wrong

This is slower than asking one strong model.

Some people will hate the manual handoffs. Some decisions do not need this much structure. A bad prompt can still produce bad deliberation. A human can still ignore the uncomfortable part of the synthesis.

Files can become ceremony.

A checklist can become theater.

A council can become a more elaborate way to hear what you already wanted to hear.

I do not think council.md magically solves that.

But I do think it makes the failure easier to see.

And for me, that is the point.

check out council.md at **https://github.com/anshulLuhsna/council.md**