Your AI Agent Has Amnesia. Karpathy’s LLM Wiki and Google’s OKF Are the Cure.

Every session starts from zero. Every query rediscovers the same knowledge from scratch. Two ideas published three months apart finally fix that — and together they’re more powerful than either one alone.

By Sanket Upadhyay | Data Scientist

There is a problem so familiar in AI engineering that most teams have stopped noticing it.

You build an agent. You connect it to your database, your documents, your APIs. It answers questions. It gets things done. And then the session ends, and the next session starts, and the agent has forgotten everything. Not just the conversation — everything. What your system does. What your metrics mean. Why certain tables exist. What happened the last time something broke.

Every session starts from zero. Every query re-derives understanding from raw documents. Every agent, every tool, every model — all of them waking up every morning with no memory of yesterday.

This is not a small inconvenience. It is the single biggest gap between an AI agent that feels impressive in a demo and an AI agent that is genuinely reliable in production. And it has been unsolved long enough that most teams have built elaborate workarounds for it — massive system prompts, exhaustive context dumps, repeated re-explanations — and called the result production-ready.

In April 2026, Andrej Karpathy published a pattern that addressed the root cause. In June 2026, Google Cloud published a specification that turned that pattern into a standard. Understanding both, and how they connect, changes how you think about building agent systems.

The Problem with How We’ve Been Thinking About Memory

The dominant solution to the “agent doesn’t know your system” problem has been RAG — Retrieval-Augmented Generation. The mechanics are familiar: you take your documents, split them into chunks, convert those chunks into vectors, store them in a vector database, and at query time, retrieve the most similar chunks and pass them to the model as context.

RAG works. For the right problems, it works extremely well. But it has a structural flaw that gets harder to ignore the longer you build with it:

RAG is stateless. Every query is a new search. The model retrieves relevant chunks, reasons over them, produces an answer, and forgets everything. Next query: start over. The model has no accumulated understanding of your domain — it has access to your documents, which it re-reads from scratch every single time.

Ask a simple question and RAG returns the right chunk and everything’s fine. Ask something that requires synthesizing five documents — understanding how a metric is calculated, what tables feed it, what breaks it, and how it connects to three other concepts — and RAG has to piece together disconnected fragments from a cold start. Every time. Because nothing accumulated.

Karpathy described this precisely: “Most people’s experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chunks at query time, and generates an answer. This works, but the LLM is rediscovering knowledge from scratch on every question. There’s no accumulation.”

His proposed fix: stop treating knowledge as something to retrieve. Start treating it as something to compile.

Karpathy’s LLM Wiki: Knowledge That Compounds

On April 3, 2026, Karpathy posted a tweet describing how he now uses LLMs to build personal knowledge wikis instead of just generating code. The post got 16 million views — an unusually large number for something that, at its core, is a description of a folder structure. The follow-up GitHub gist hit 5,000 stars within days.

The number is worth pausing on. 16 million views for a markdown file. That doesn’t happen because the idea is clever. It happens because it solves a problem every knowledge worker, every AI engineer, and every serious builder has been carrying around for two years and nobody had named precisely.

The LLM Wiki is not a product. Karpathy explicitly published it as what he called an “idea file” — a pattern written in plain prose, designed to be pasted into an LLM agent so the agent can build a version tailored to your specific needs. His framing: “In the age of LLM agents, sharing an idea is more valuable than sharing code, because the other person’s agent customizes and builds it for your specific needs.”

The pattern itself is built around an analogy from software engineering that makes it immediately intuitive: compilation.

When you write source code, a compiler doesn’t execute it line by line every time you run the program. It transforms the source into an optimized artifact — once — and every subsequent run uses the compiled version. The compilation is expensive. Running the compiled output is fast and consistent.

RAG is like executing source code directly every time. The LLM Wiki is like compiling it first.

Instead of retrieving raw documents at query time, the LLM incrementally builds and maintains a persistent wiki — structured, interlinked markdown files that sit between you and your raw sources. Knowledge is compiled once and kept current, not re-derived on every query. Each time you add a new source, the agent reads it and updates the wiki. Each time you ask a question, the agent reads the wiki — not the raw documents.

The result is something Karpathy calls a compounding artifact. The wiki gets richer every time you add a source. It gets deeper every time you ask a question and file the answer back in. It develops cross-references that no retrieval system would surface because no retrieval system knows which concepts relate to which. Contradictions between sources get flagged and resolved in the wiki rather than silently confusing the model at query time.

The Three-Layer Architecture

The LLM Wiki has three layers, and understanding each one is important because they have different owners with different rules.

Layer 1: Raw sources. Your documents, PDFs, articles, research papers, transcripts — whatever the raw input material is. This layer is immutable. The agent reads it but never modifies it. You curate what goes in here. The agent does not.

Layer 2: The wiki itself. LLM-generated and LLM-maintained markdown files: entity pages for specific concepts, summary pages for sources, comparison pages, cross-reference lists, contradiction logs. The LLM owns this layer entirely. When you add a new source, the agent reads it, updates relevant entity pages, creates new ones if needed, adds cross-references, notes contradictions — touching 10 to 15 files in a single ingest pass. Humans read it but don’t write it. The LLM writes it but doesn’t invent it. Everything in the wiki traces back to sources in Layer 1.

Layer 3: The schema. A CLAUDE.md or AGENTS.md file that tells the agent how the wiki is structured, what naming conventions to follow, what workflows to run on ingest versus query, and what maintenance operations to perform on a lint pass. This is what turns a generic language model into a disciplined wiki maintainer rather than a creative writer who invents plausible-sounding facts.

Karpathy’s own description: “Obsidian is the IDE. The LLM is the programmer. The wiki is the codebase.”

The tooling he recommends: Claude Code as the agent, Obsidian as the viewer (for graph visualization and markdown rendering), and qmd — a local BM25/vector hybrid search tool built by Shopify's Tobi Lutke — as the search layer when the wiki grows large enough that reading the index isn't enough.

What Happens Inside the Wiki

The three core operations are ingest, query, and lint — and each one works differently than you might expect.

Ingest is where most of the token cost lives. When you drop a new source into the raw folder and tell the agent to ingest it, the agent reads the source, writes a source summary page, then scans the existing wiki to find related entity and concept pages that need updating. A 10-page paper might trigger updates across 12 wiki files. The ingest is expensive — roughly 5 to 8 times the source’s own token count for a full pass. But you pay it once. Every subsequent query against the compiled knowledge is cheap.

Query is where the payoff shows up. The agent reads the relevant wiki pages — starting from the index, following links — rather than searching raw documents. The quality difference from RAG is significant. The model isn’t reading a random chunk from page 14 of a PDF. It’s reading a pre-synthesized, cross-referenced entry that already integrates everything the system has ever learned about that concept from every source ever ingested. Then the answer can be filed back into the wiki. Future sessions benefit immediately. This is the compounding principle.

Lint is what keeps the wiki from rotting over time. As any wiki grows, contradictions appear. A source from March says one thing; a source from May says another. Entity pages develop orphaned links. Cross-references go stale. Lint scans for these issues and flags or resolves them. Karpathy recommends running it weekly or after large ingest batches. Think of it as a CI/CD pipeline for knowledge.

The Practical Ceiling — and Where RAG Still Wins

The LLM Wiki is not a replacement for RAG. This distinction matters enormously and gets blurred constantly in coverage of this topic.

The LLM Wiki’s practical ceiling is roughly 50,000 to 100,000 tokens for a single-context approach — about 150 to 200 dense wiki pages. Past that point, the agent can’t load the full wiki into context efficiently and the pattern starts breaking down.

RAG doesn’t have this ceiling. For large-scale enterprise document archives — thousands of PDFs, constantly updated, with no stable semantic structure — RAG is still the right architecture. The retrieval model handles scale that pre-compilation can’t.

The strongest architecture for mid-scale use cases combines both: a compiled wiki for stable core knowledge, plus RAG for dynamic or overflow content. The wiki handles “what does this metric mean and why does it break” — stable, curated, compounded over time. RAG handles “what specifically happened on June 23rd” — dynamic, date-specific, drawn from operational logs that change daily.

This is precisely the stack your friend’s OKF article demonstrated: OKF (the compiled layer) plus RAG (the dynamic layer) plus LLM reasoning over both. Neither is complete alone. Together they cover the full spectrum from “stable understanding” to “specific recent facts.”

Google’s OKF: Karpathy’s Pattern, Formalized

On June 12, 2026 — ten weeks after Karpathy published his gist — Google Cloud’s Data Cloud team, led by Sam McVeety and Amir Hormati, published the Open Knowledge Format v0.1.

The framing in the announcement was explicit: OKF formalizes the LLM Wiki pattern that Karpathy described into a portable, vendor-neutral standard. One team in the community called it “Google formalizing the power of the LLM Wiki.” That’s an accurate description.

What OKF adds is not a new idea. It’s interoperability. Karpathy’s pattern works brilliantly as a personal or team-level system, but every team that implemented it built it differently — different folder structures, different frontmatter conventions, different naming schemes. Knowledge compiled by one team’s wiki couldn’t be consumed by another team’s agent because there was no standard for what a knowledge file was supposed to look like.

OKF fixes that. It is to the LLM Wiki what JSON was to data exchange — a minimal, agreed-upon structure that makes producer and consumer independent of each other.

The format is genuinely simple. Every OKF file is a markdown file with a YAML frontmatter block at the top. The only required field is type. Everything else — title, description, resource, tags, timestamp — is optional but useful. The body of the file is plain markdown: the actual knowledge, written by a human who understands the system, in whatever structure makes sense for that concept.

A collection of OKF files is called a bundle. A bundle is just a folder. You organize it like any codebase: metrics/, tables/, agents/, runbooks/, apis/. Files link to each other with ordinary markdown links. The directory becomes a graph of related knowledge that an agent can navigate by following links, the same way a human navigates Wikipedia.

No vector database. No embedding model. No chunking pipeline. No SDK. No central registry. No proprietary service to call. Just files in a folder, readable by any tool that can open a markdown file.

Why the Simplicity Is the Point

This is the design choice worth dwelling on, because it’s the one that distinguishes OKF from every previous attempt to solve the agent knowledge problem.

The most common failure mode in enterprise knowledge systems is that the infrastructure becomes the maintenance burden. You build a knowledge graph, and now you’re maintaining a knowledge graph — schema migrations, consistency checks, dependency upgrades, access control. The knowledge gets buried under the infrastructure keeping it alive. That’s why enterprise wikis go stale and RAG pipelines get polluted with outdated documents. The maintenance cost grows faster than the value returned.

OKF’s design philosophy is the same as git, markdown, and JSON — formats that lasted because they required nothing to use. Swapping producers or consumers doesn’t break the knowledge. No proprietary account is required to read a file. The value scales with adoption breadth, not vendor lock-in.

Karpathy’s gist made the same observation about why wikis fail: “Humans abandon wikis because the maintenance burden grows faster than the value. LLMs don’t get bored, don’t forget to update a cross-reference, and can touch 15 files in one pass.” OKF gives the LLM a standard structure to maintain, so the wiki it builds can be read and extended by any agent, any tool, any team — not just the one that built it.

The Full Stack: Where LLM Wiki, OKF, and RAG Each Belong

Pulling all of this together, the complete picture of a production agent knowledge layer looks like this:

Your raw data sits at the bottom: databases, documents, logs, APIs, operational records.

Above that, two compiled layers run in parallel.

OKF / LLM Wiki handles the stable layer: the curated, human-verified knowledge about what things mean — metric definitions, table schemas, agent descriptions, runbooks, API contracts. This knowledge doesn’t change often. It’s written by people who understand the system. It’s maintained by an LLM that keeps cross-references current and contradictions flagged. It answers: “What does this term mean, how is it calculated, what breaks it, and how does it connect to these other concepts?” This knowledge costs nothing to read at query time because it’s already compiled. It cannot hallucinate because a human wrote it.

RAG handles the dynamic layer: the specific, time-bounded facts that change constantly — what happened on a particular date, what a specific document said, what errors appeared in last night’s logs. This answers: “What specifically occurred on Tuesday? Which documents failed? What did the incident report say?” RAG is designed for this — scale, dynamism, recency. The compiled wiki is not.

The LLM sits above both and reasons over both simultaneously. The stable layer gives it understanding. The dynamic layer gives it facts. The reasoning is what connects them into an answer. Take away either layer and the answer degrades: no stable knowledge and the model has facts but no framework to interpret them; no dynamic retrieval and the model understands the system perfectly but doesn’t know what happened.

The reason most agent answers sound plausible but wrong is not that the model is bad. It’s that one of these layers is missing. The model has facts without understanding, or understanding without facts, and it fills in the gap with confident inference that happens to be incorrect.

What This Means for What You Build

The practical implication is more actionable than most architectural advice.

If you are building an AI agent that needs to understand how your organization’s systems work — your metrics, your data pipelines, your APIs, your operational procedures, your business logic — that knowledge belongs in an OKF bundle or an LLM Wiki, not in a system prompt you’re constantly editing and not in a RAG pipeline you’re hoping will surface the right chunk.

The implementation cost is lower than almost any alternative. A knowledge folder with ten markdown files covers most of the stable knowledge a focused agent needs. A 20-line Python loader reads those files and assembles the relevant context. Total added context at query time: 500 to 700 tokens of zero-hallucination, human-verified facts. Compare that to the alternative — 20,000 tokens of RAG-retrieved chunks that may or may not contain the right information and definitely don’t know how the concepts connect.

The distinction Karpathy draws is worth closing on. The question we’ve been asking is “how do I make the model smarter?” The question that actually matters in 2026 is “how do I structure the information the model has access to?” A smarter model with the same fragmented, stateless context produces smarter-sounding wrong answers. A baseline model with well-structured, compiled, stable knowledge produces correct ones.

The wiki doesn’t make the model smarter. It gives the model something worth being smart about.

This article draws on Andrej Karpathy’s LLM Wiki gist (gist.github.com/karpathy/442a6bf555914893e9891c11519de94f, April 2026), the LLM Wiki v2 extension by rohitg00, and the Open Knowledge Format v0.1 specification published by Google Cloud on June 12, 2026.

Tags: AI Agents · LLM · RAG · Knowledge Management · Machine Learning · Software Architecture · Google Cloud · Andrej Karpathy · Data Science