Everyone Building RAG Pipelines in 2026 Is Already Behind.

A few weeks ago I wrote about a $750,000 AI system I helped build. Not a chatbot, but a pipeline that took a huge pile of messy source documents (scanned contracts, inconsistent PDFs, spreadsheets written by a dozen people over a dozen years) and turned them into structured, professional documents, section by section, against a fixed template.

The engineering was real. We normalized everything through an ingestion pipeline, embedded it into a vector store, and then solved the hardest part: matching the right chunks of source material to the right sections of the output, with a hybrid retrieval approach combining HyDE and BM25. Semantic search for meaning, keyword search for the exact terms and figures that semantic search tends to blur past. Once that mapping was right, the writing step was almost boring by comparison, which was the whole point.

I still stand behind that architecture. But there’s a question I didn’t ask at the time, and I think a lot of teams building “serious” RAG systems don’t ask it either: after all that engineering, where does the knowledge actually live?

The part nobody puts in the case study

Here’s the honest answer for my project, and probably for most RAG systems built in the last three years: the knowledge lived inside a vector store, reachable only through the retrieval code we wrote to query it.

That’s not a criticism of the architecture. It’s just a fact about what a vector store is. It’s an index, not a document. You can’t open it in a text editor. You can’t diff it in git and see what changed. You can’t hand it to someone on another team and have them read it. To get anything human-readable back out, you have to go through the same pipeline that put it in, or write a new one that speaks the same embedding format. The knowledge was real, and it was expensive to build, and it was completely locked inside the system that produced it.

At the time, that felt like an acceptable cost of doing serious AI work. Context windows were small, models were expensive per token, and retrieval-augmented generation was the correct answer to a real constraint: you couldn’t fit the whole knowledge base into a prompt, so you fetched the relevant slice and fed only that. Graph RAG pushed the idea further, building an explicit graph of entities and relationships so a model could reason across connections instead of isolated chunks. All of it was solving the same problem: how do you get the right knowledge in front of a model that can’t hold everything at once.

What none of it solved was portability. The knowledge stayed gated behind whichever framework, vendor, or pipeline happened to build the index.

A quieter idea that’s been sitting right in front of us

I came across a piece recently from the team at Formaly that names something I think a lot of us have been circling without saying out loud: a lot of what we built to make knowledge “AI-ready” didn’t need to be that complicated. Google Cloud recently put out something called the Open Knowledge Format, OKF, still at version 0.1, and the idea is almost aggressively unglamorous. It formalizes a pattern a lot of teams were already doing by instinct: keeping an internal wiki as plain markdown files, one file per concept, organized in a sensible folder hierarchy, with a small amount of structured metadata at the top of each file. A dataset gets a file. A table gets a file. A metric, a runbook, an API, each one is just a markdown document with some frontmatter, sitting in a directory you could open in any text editor or browse straight on GitHub.

That’s it. No proprietary catalog, no SDK you have to install to read your own data, no service you have to query to find out what you know. It’s a folder. You can put it in git. You can grep it. A human can read it top to bottom, and so can an agent, with no translation layer between the two.

Read on its own, that sounds almost too simple to matter. A spec with one required field isn’t a technical achievement; plenty of teams have quietly built the same thing for themselves without calling it a standard. But that’s exactly why I think it matters more than it looks like it does at first glance. The value was never going to be in the format’s cleverness. It’s in the fact that it’s a shared agreement. The moment “how we store our knowledge” stops being a proprietary decision made by whichever pipeline got built first, the knowledge stops being gated by accident.

What this would have changed about the $750K build

I keep coming back to our ingestion pipeline when I think about this, because OKF isn’t really competing with what we built. It’s answering a question we never asked while we were building it.

Our vector store was the right tool for what it did: fast, meaning-aware retrieval over a huge, messy corpus at the moment of authoring a document. But it was never meant to be the durable record of what the system knew. If a new team had come in six months later and asked “what does this system actually understand about the source material,” there was no good answer that didn’t involve re-running our retrieval code. The knowledge and the pipeline were the same object.

If the ingestion stage had also produced an OKF-style bundle alongside the embeddings, one markdown file per contract, per report, per key term, organized by concept instead of buried as an anonymous chunk ID, the vector store would still have done its job for fast retrieval during authoring. But there would also have been a second, plainer copy of the knowledge sitting in files: readable by a human doing QA, diffable in git as source documents got updated, and usable by an entirely different tool six months later without anyone having to reverse-engineer our embedding pipeline first. The auto-mapping step, the part I called the real engineering value in that project, would have had a much more legible source to map from.

RAG would have stayed exactly what it’s good at: fast, ranked retrieval at generation time. OKF would have covered what RAG was never designed to cover: a durable, human-readable, tool-agnostic record of what the system actually knows, independent of whichever pipeline happens to be running this year.

Where I think this is actually going

I don’t think OKF replaces retrieval-augmented generation, and I don’t think that’s the right way to read it. Vector search and hybrid retrieval solve a real problem; you still need fast, ranked access into a large corpus at the moment a model is generating something. OKF solves a different, quieter problem: what does the knowledge look like when nothing is querying it. Sitting on disk. Owned by whoever wrote it. Readable by whatever shows up next.

The pattern I’d bet on for the next wave of these systems looks like both layers at once: a vector store or graph doing retrieval at generation time, sitting on top of a plain, portable, markdown-based knowledge layer that isn’t owned by that retrieval system at all. The $750K version of this build spent almost all its engineering budget on the first layer. The next version of it, I suspect, spends real effort on the second one too, not because it’s technically hard, but because it’s the difference between knowledge you built and knowledge you can still read.

If you want to see the architecture from the $750K project in more detail, the full ingestion pipeline, the hybrid retrieval setup, and a worked example, I wrote it up separately, linked in my other article.