Folders Over Agents + ICM: A Deep Breakdown

Before I get into it this post is heavily inspired by the work of Jake Van Clief. I came across his ideas and honestly it shifted how I think about building with AI. If you don’t follow him already, you should. Everything I’m breaking down here traces back to his original thinking, so credit where it’s due.

The Core Problem

Most people building AI workflows today make one critical mistake: they put the logic inside the agent.

This looks like:

• Massive system prompts stuffed with instructions
• LangChain/CrewAI “agents” with hardcoded reasoning chains
• Workflows that say “if the AI returns X, do Y”

The problem? The AI is a moving target. GPT-4 → GPT-4o → o1 → o3 all behave differently. Every model upgrade potentially breaks your entire system because your logic was glued to the model’s behavior.

The Philosophy Flip

The AI should be stateless and replaceable. Your structure should be stateful and permanent.

Think of it like a database analogy:

The Five ICM Layers

Layer 1: Identity

This is not a system prompt. It’s a persistent, structured definition of what this workflow is.

Think of it as a who-am-i.md file that answers:

• What is the purpose of this system?
• What persona/role does the AI take here?
• What does success look like?

Why it matters: Every AI session starts fresh. Without a persistent Identity layer, you re-explain the context every time wasting tokens and getting inconsistent behavior.

Layer 2: Routing

This is arguably the most underrated layer. It’s a navigation map for the AI.

Instead of dumping 50 documents into context and hoping the AI figures it out, the Routing layer says:

“If the task is X, read these files. If the task is Y, go here instead.”

This is basically a context-aware index. The AI doesn’t scan everything, it follows the route to exactly what it needs.

Why it matters: Directly slashes token usage and hallucinations. A confused AI is usually a poorly routed AI.

Layer 3: Stage Contract

This defines the rules of engagement for a specific task stage.

Examples:

• “At this stage, only produce structured JSON. No prose.”
• “Do not ask clarifying questions. Execute based on available context.”
• “Output must follow this schema: {finding, severity, evidence, recommendation}"

It’s like a typed function signature the AI knows exactly what input it gets and exactly what output format is expected. No ambiguity.

Why it matters: Agents fail most often at boundaries between stages. The Stage Contract removes that ambiguity entirely.

Layer 4 : Reference Material

This is the knowledge base the actual structured data the AI reads.

Key point: this isn’t raw dumps of information. It’s organized, chunked, and labeled so the AI can navigate it efficiently.

Examples:

• A targets/ folder with per-target recon notes (in your context, think RECON HYDRA outputs)
• A findings-db/ with past vulnerability reports sorted by type
• A methodology/ folder with step-by-step process docs

The folder structure itself communicates meaning. critical-cves/ vs low-severity/ tells the AI how to weight things without you having to say it.

Layer 5 : Working Artifacts

These are the outputs the AI actually produces — but they’re also inputs for the next stage.

The key insight here is that artifacts are version-controlled and structured:

• draft-report-v1.md → AI reviews → draft-report-v2.md
• recon-findings-raw.json → AI enriches → recon-findings-enriched.json

The AI never starts from scratch. It always picks up a Working Artifact, applies the current Stage Contract, and produces the next version.

Why it matters: This is how you build systems that compound over time. Each artifact is a checkpoint. If the AI fails or a model changes, you resume from the last clean artifact not from zero.

Why This Is Powerful for Engineers

The real insight is that ICM is model-agnostic by design.

Today you use Claude Sonnet. Tomorrow you swap in Gemini 2.5. Next year something better comes out. Your folder structure doesn’t change. Your routing doesn’t change. Your Stage Contracts don’t change. You just point a smarter reader at the same library and it performs better automatically.

Compare this to a LangChain agent that has 800 lines of prompt-engineering glue code built around GPT-4’s specific quirks. That thing breaks every few months.