We deleted fifty years of information retrieval and called it RAG

Chunking, embeddings, hybrid search, reranking, eval. Every one already had a name in information retrieval. We didn’t lose it, we just stopped reading.

Image by Andrew Stutesman on Unsplash

Earlier this year, I lost a run of late nights to chunk sizes for a finance client’s document pipeline. 256 tokens, then 512, then 1024, then 512 with a 64-token overlap, then 384 because some thread swore by it. I had a notebook open, a little eval harness I was proud of, and two chat windows going in other tabs. I was asking ChatGPT and Claude for a practical chunking strategy the way you’d ask a senior engineer, then cross-checking them against whatever the internet had posted that month. It felt like tuning a frontier. I’d land on a winner and a chart, and go to bed pleased with myself.

The trouble showed up as the corpus grew. The more documents I loaded, the more points piled into the vector store, the worse retrieval got. Something that looked sharp on a small sample turned mushy at scale. It handed back passages that sat close enough in vector space and were useless in fact. So I tuned the chunking harder, because chunking was the knob I knew how to turn. I kept at it for days, certain the next setting would be the one that fixed it. Every run moved some number, and I took that for progress. None of them touched the actual problem.

Then one day, another engineer on the team, looking over the config, asked with no irony at all: “what’s BM25?” I didn’t have a clean answer ready. I knew it was old. I had the foggy sense that it was “just keyword search,” the thing we’d all supposedly moved past. What I didn’t understand yet was that I’d been standing inside a fifty-year-old conversation and hadn’t bothered to learn the names of anyone already in the room.

That’s the confession.

The vertigo

It took me an embarrassingly long time to see it clearly. Almost every piece of what I do when I “build RAG” already has a name, and most of those names have dates older than I am. Some are older than the people selling the ideas back to me with a pitch deck. Cheap shot. Most of them don’t know the lineage either, same as I didn’t.

I’m not the first to notice this. Jo Kristian Bergum, who spent years as a distinguished engineer at Vespa, put it plainly in a 2024 *post* on the retrieval side of RAG:

Hybrid retrieval and ranking pipelines allow you to combine signals from unsupervised methods (such as BM25) with supervised methods (such as neural rankers).

When you retrieve documents to stuff into a prompt, you are doing information retrieval, and the work reaches far past one cosine score into rewriting, fusion, reranking, phased retrieval, all of it.

Daniel Tunkelang, who did relevance for a living back when that was an unglamorous job nobody bragged about, *keeps pointing out* that retrieval didn’t die. It went underground and quietly kept running the world. They’re right. But “RAG is just information retrieval (IR)” is a one-liner, and one-liners are cheap. What moved me from nodding along to actually changing how I work was reading the receipts.

Let me walk you through the receipts. Not to dunk on anyone. (I keep saying “we” and “the field” in this piece. I mean me. I did every one of these things, some of them last quarter.) Think of it as the map I wish someone had handed me, the one that didn’t start in 2020 with the first fifty years torn out.

The mapping

Chunking first, because that’s where I embarrassed myself, so it’s only fair.

The idea that you retrieve passages instead of whole documents is old enough to have a midlife crisis. Salton, Allan, and Buckley were writing about passage retrieval in 1993. Callan studied passage-level evidence in 1994. Kaszkiel and Zobel published a paper in 1997 literally titled “Passage Retrieval Revisited,” and what they were revisiting was already old enough to revisit. They were already testing how passage size and segmentation change retrieval. Plain fixed-length passages held up surprisingly well. Which is the thing I spent a night re-deriving, badly, with a worse method.

Marti Hearst’s TextTiling, also 1997, segments text where the topic actually shifts instead of at an arbitrary token count. Every “semantic chunking” startup is selling that back to us right now, and good for them, but the math had a citation before I owned a computer.

Then the holy of holies, the embedding. Vector search feels like the genuinely modern part, the thing that separates us from the keyword peasants. Documents and queries as vectors, similarity as the cosine between them. That’s Salton, Wong, and Yang in 1975. The vector space model. Weighting rare terms more heavily, which your embedding model now does for free in its weights, is Karen Spärck Jones in 1972. And the move everyone treats as the soul of the whole thing, matching on meaning rather than surface words to get past vocabulary mismatch, is Latent Semantic Analysis: Deerwester and colleagues in 1990, running an SVD over the term-document matrix to pull out latent structure. We swapped the SVD for a transformer, and a lot of us quietly decided that made the idea ours. To be fair to us, the transformer is not a cosmetic swap. It reads a word differently depending on the words around it, which a single static LSA vector never could. But the move underneath, meaning over surface form, is the old one. That one was Deerwester’s.

Hybrid search is the current darling, where you blend lexical and dense scores and feel clever. The fusion method baked into Elasticsearch, OpenSearch, Weaviate, and Azure AI Search is reciprocal rank fusion, which is Cormack, Clarke, and Büttcher at SIGIR 2009. Their paper has a detail I think about more than is healthy: a dead-simple fusion with a constant they set to 60 and never bothered to tune beat the trained learning-to-rank methods it was up against. The thing nobody tuned beat the things someone trained. And combining the outputs of multiple retrieval systems was already routine in 1994, when Fox and Shaw were doing it under names like CombSUM and CombMNZ.

Query rewriting is the one that should make you put your coffee down. HyDE, where you have the model hallucinate a fake answer document and embed that instead of the question, on the theory that a plausible wrong answer lives nearer the right documents than the question does. Genuinely clever.

It’s also Rocchio, 1971.

Relevance feedback nudges the query vector toward documents that look relevant and away from the ones that don’t, and pseudo-relevance feedback, Croft and Harper in 1979, does it blind by assuming your top results are good enough to learn from. HyDE is, in spirit, neural pseudo-relevance feedback: the same move Rocchio and Croft were making decades ago, now run through a transformer. And here’s the part that stung when I went looking: the HyDE paper never cites Rocchio. So the amnesia isn’t only in the tutorials, the threads, and my own notebook. Sometimes it’s in the papers too. Better still, the old literature already named the failure that bites every HyDE setup in production: query drift. If your top results are bad, expanding from them makes everything worse, which is exactly why the old hands ran feedback once and stopped. We run it in agentic loops and act surprised when retrieval wanders off into a field somewhere.

I’ll go quicker now, because the shape is the shape. Your cross-encoder reranker, the thing you bolt onto first-stage retrieval to clean up the top results, is a two-stage cascade. Wang, Lin, and Metzler formalized cascade ranking in 2011, and learning-to-rank had RankNet by 2005. The BM25-then-cross-encoder pipeline I’ve shipped four times this year is a textbook two-stage cascade, renamed.

And then evaluation, which should sting most, because it’s the part we do worst. The whole practice of fixing a corpus, writing test queries, collecting human relevance judgments, and scoring systems offline is the Cranfield paradigm. Cleverdon and colleagues, 1966. TREC, run out of NIST, industrialized it from 1992 on, and Voorhees and Harman wrote the book on it in 2005. They worked out pooling decades ago so you could judge results without reading the entire collection. They settled on metrics with real meaning, like MAP and nDCG. Most RAG “eval” I’ve seen, mine included, is a tiny undocumented version of this with no relevance judgments at all, scored by an LLM judge nobody validated, on a test set quietly assembled from the queries that already happened to work.

Reinvented, often worse

That last move has a name in the old world, and it isn’t a kind one. Build your eval set out of queries you already know retrieve well and you’ve sampled on the dependent variable. You’ll pass your own eval, ship something that face-plants on everything you didn’t think to test, and then file the failures under “hallucination,” because you have no instrument pointed at the real cause. And the rigged eval is not useless. It catches a real regression on the queries you already care about. It just goes quiet on the ones you never thought to ask, which are the ones that break in production. Mine broke on the first question that fell outside the set I built it from, a number over a threshold, a date range, the ordinary shape of a finance query I had never thought to test.

The receipts here are quantitative, which means I don’t get to argue with them.

Start with the assumption that sent me chasing chunk sizes instead of testing a baseline at all: that dense retrieval simply beats lexical. BEIR, from Thakur and colleagues in 2021, put ten systems against eighteen datasets and found the result that belongs stenciled inside every RAG repo. In-domain performance does not predict out-of-domain performance. BM25 trailed neural models by seven to eighteen points on the data they were trained on, then turned around and beat most of them the moment you left that distribution. The best dense bi-encoder in the study won on eight of the eighteen datasets. Eight. My corpus was a niche finance pile, exactly the specialized, out-of-distribution kind of text BEIR is warning about, and the assumption that dense would beat lexical anyway was so total that I never ran the lexical baseline against it. The paper that would have told me to had been sitting there for four years.

I should hedge that one, because the frontier moves. A few commercial leaderboards now claim dense retrieval beats BM25 by fifteen to twenty-five percent on BEIR. Maybe. Those are vendor numbers on vendor terms, and I read them the way I read any benchmark run by the people selling the thing. The peer-reviewed position is still the boring correct one: BM25 is a strong baseline, hybrid is the safe default, and the day that stops being true I’d like to see it demonstrated by someone with nothing to sell.

Then there’s the older, sadder pattern of beating up weak baselines and calling it progress. Jimmy Lin wrote a piece in 2018 with the gloriously grumpy title “The Neural Hype and Comparisons Against Weak Baselines,” arguing that a lot of neural retrieval “wins” were really wins over a BM25 the authors couldn’t be bothered to configure. The 2019 follow-up went through years of papers on a standard test collection and found no upward trend in effectiveness at all. Their line: the best reported results were from a decade earlier, and nothing recent came close. So now, every time a RAG benchmark shows vector search crushing “keyword search,” the first thing I want to see is the BM25 config. The most common result in this field is a strong old method losing to a lazy implementation of itself.

The one that made me laugh, in the bad way, is position bias. “Lost in the Middle,” Liu and colleagues, 2023, showed that models use information best at the start and end of their context and worst in the middle. A U-shaped curve. The numbers aren’t subtle: in their multi-document QA setup, GPT-3.5-Turbo, with the answer buried in the middle of twenty or thirty retrieved documents, scored below its own closed-book number of 56.1 percent. Adding the retrieved context made the model worse than answering from memory. In a synthetic test, the weakest models fell under forty percent when the target sat in the middle of the inputs, and the U-shaped pattern showed up across GPT-4, Claude, and a handful of open models. We greeted this as a fresh discovery about a strange new kind of machine.

Joachims and colleagues ran an eye-tracking study in 2005 on how people read ranked search results, and followed it up in 2007. Their attention pooled at the top of the list. Users kept clicking the top-ranked link even when something lower was more relevant. The literature came to call it a trust bias. The order you present evidence in changes how that evidence gets used, independent of whether it’s any good. We knew that about humans eighteen years before we “found” it in models.

I want to be careful here, because the analogy isn’t exact, and pretending it is would make me the exact thing I’m complaining about. Classical position bias is monotonic, strongest at the very top and fading down the list. The model effect is U-shaped, with the dead zone in the middle and a recency bump at the end. Different shapes. Same family of problem. And the model version is itself moving under us: the follow-up work, some of it from 2025, finds that middle penalty drifting around as inputs stretch toward the context limit, not reliably going away. So don’t quote “Lost in the Middle” as scripture either. The durable lesson is that order distorts use. The exact U-curve is a moving target, which is its own argument for measuring instead of memorizing.

Where it actually is new

I resisted writing this section. Conceding ground weakens the dunk, and the dunk is more fun. But if I stop at the receipts I’ve written a clever takedown, and a clever takedown is a way of being wrong at length. The old maps do run out, and where they fail is what’s actually worth your attention.

The deepest break is who, or what, is reading. For fifty years the consumer of retrieval was a person scanning a list, so relevance got defined as something a human could judge at a glance. Now a generator does the reading, and recent work names the shift without flinching: this is search for an AI, not search for a person. Which makes relevance environment-dependent, and genuinely hard to define in advance. It moves the target.

It moves it in ways that break the old assumptions outright. Cuconasu and colleagues ran the test in a 2024 SIGIR paper, “The Power of Noise.” One half of the result is unsurprising: a retriever’s highest-scoring documents degraded the output when they were topically close but didn’t carry the answer. The other half is not. Adding random, unrelated documents improved accuracy by as much as thirty-five percent. Cranfield assumed a document’s relevance was a property of the document, independent and additive. In RAG a passage’s worth depends on everything else in the context and on how the generator stitches it together. The atom of relevance changed, and most of our metrics still weigh the old atom.

Then there’s a failure mode with no classical ancestor, because the old systems couldn’t commit it if they tried. Classical IR could rank badly. It could not lie. It returned documents that existed; it never wrote a fluent, confident paragraph that the documents flatly contradict. Faithfulness, whether the generated text is actually supported by what you retrieved, is a new axis, and tools like RAGAS exist as of 2023 to measure something the entire prior field never had to name. Agentic retrieval is partly new too. The individual search call is ancient, but a model that decomposes a question, fires its own sub-queries, reads what comes back, and re-plans is an autonomous control loop wrapped around classical retrieval, and the loop is the new part. Khattab’s DSP and DSPy work is the serious version of that, and it’s no accident it comes from someone who knows the old literature cold.

And the scoreboard. nDCG and MAP and MRR measure how good a ranked list is for a human reading down it. They say nothing about whether the final generated answer is correct, complete, and grounded. Which is why you can’t just port the old metrics over and declare yourself done.

The people it happened to are still in the room

The researchers who built all of this didn’t retire to a beach. They’re right here, and several of them are quietly folding RAG into the evaluation tradition it spent two years ignoring.

Jimmy Lin maintains the tooling that makes a properly tuned BM25 a single command, Anserini and Pyserini, so the weak-baseline excuse is dead on arrival now. After the grumpy “neural hype” pieces, he turned around in 2019 and published one called “The Neural Hype, Justified! A Recantation,” conceding that with strong baselines, the neural methods of the BERT era really did show gains in low-resource settings. He’d argued the field was overhyping, looked at better evidence, and changed his mind in public and in print, and he turned out to be partly right about the hype anyway. That’s the posture worth having, going both directions: the old guard updating toward the new, the new generation reading far enough back to know what it’s updating from.

Bhaskar Mitra and Nick Craswell wrote an introduction to neural IR in 2018 that deliberately set the neural models inside the classical tradition, with BM25 as the baseline you have to respect. Then they ran the TREC Deep Learning track from 2019 on, and benchmarked the entire neural era using the same pooled, reusable test collections that Cranfield and TREC had spent decades sharpening. So the revolution got scored on the old field’s instruments, by the people who built them. It held up. Which tells me the rigor was transferable the whole time, if anyone wanted it.

Ellen Voorhees and Donna Harman edited the book on TREC evaluation. Voorhees spent years on the uncomfortable fact that human relevance judgments disagree with each other, which is the exact problem now facing everyone who wants to swap human judges for an LLM. When the IR community looked at doing that, they wrote it up as a real argument. Named researchers warned it could quietly corrupt the science. The pilot data had model judgments tracking human ones at a Kendall’s tau around 0.86. Which sounds reassuring, except a single correlation says nothing about where the two disagree, and the disagreements that matter are the borderline documents that decide which system wins.

And now there’s a track for it. NIST launched a TREC RAG track in 2024, three hundred and one topics over a deduplicated MS MARCO collection of a hundred and thirteen million passages. Anyone who thinks evaluation is a pytest hasn’t seen the unglamorous work that goes into a test set you can actually trust. It scores systems on nugget recall, which revives Voorhees's old nugget method for the LLM era, and on citation support, which checks whether the generation is actually backed by the retrieved text. In 2025 they pushed the queries from short keyword strings toward long, messy, multi-sentence narratives, which is what real questions look like. The establishment didn't get disrupted out of existence. It spun up a track and invited RAG inside, which is a very TREC thing to do.

What we get if we read it this time

Retrieval isn’t dead. RAG didn’t change everything. I’ve come to distrust anyone selling either line, because both are ways to skip the work. What actually happened is smaller, and more useful than either slogan. We got an astonishing new way to consume retrieved text, and in the rush we threw out fifty years of hard-won knowledge about how to retrieve it well in the first place.

We can pick that back up whenever we want. The citation graph is sitting right there. The benchmarks are public. The people are alive, and in my experience weirdly delighted when someone under forty turns up with an informed question.

So I’ve started doing the boring thing. I tune the BM25 baseline before I touch the embeddings. The eval sets come from the queries I’m afraid of now, not the ones that already work. And the related-work section turns out to be where a field keeps its memory; I follow the citations backward until they stop. It doesn’t give me the hit that tuning a chunk size did. A baseline you expect to win is boring in a way that re-deriving a frontier never was. I miss it. It’s also the difference between a pipeline that demos well and one that survives a corpus nobody scraped.

The engineer who asked me “what’s BM25” wasn’t behind. He’d just been handed the same map I had, the one that starts in 2020 with the first fifty years torn out. I keep those torn-out pages on my desk now. You should find your copy. We’re going to need them, and the strange, lucky part is that we never actually lost them. We just stopped reading.

References:

The originals, grouped the way the piece walks through them. If it did its job, you’ll go read at least one instead of taking my word for it.

Passages, chunking, embeddings

• Gerard Salton, James Allan, Chris Buckley (1993). “*Approaches to Passage Retrieval in Full Text Information Systems.” SIGIR* ’93. Retrieving passages instead of whole documents, the year before the web had a search engine.
• James P. Callan (1994). “*Passage-Level Evidence in Document Retrieval.” SIGIR* ’94. Paragraph vs fixed-window passages, tested inside a real retrieval system thirty years ago. Your “chunking strategy” has a parent.
• Marcin Kaszkiel, Justin Zobel (1997). “*Passage Retrieval Revisited.” SIGIR* ’97. Ran the passage-size experiments and found plain fixed-length passages robust and effective. The result your eval harness will rediscover if you let it.
• Marti A. Hearst (1997). “*TextTiling: Segmenting Text into Multi-Paragraph Subtopic Passages.” Computational Linguistics* 23(1). Segment where the topic actually shifts, not at an arbitrary token count. Every “semantic chunking” pitch, with a citation older than the startups selling it.
• Gerard Salton, A. Wong, C.S. Yang (1975). “*A Vector Space Model for Automatic Indexing.” Communications of the ACM* 18(11). Documents and queries as vectors, relevance as the cosine between them. The operation your vector database runs at query time, written down in 1975.
• Karen Spärck Jones (1972). “*A Statistical Interpretation of Term Specificity and Its Application in Retrieval.” Journal of Documentation* 28(1). Rare terms carry more signal than common ones: idf. Your embedding model learned this for free; she derived it.
• Scott Deerwester, Susan Dumais, George Furnas, Thomas Landauer, Richard Harshman (1990). “*Indexing by Latent Semantic Analysis.” JASIS* 41(6). Matching on latent meaning instead of surface words, via an SVD over the term-document matrix. Swap the SVD for a transformer and you have the idea everyone thinks they invented in 2020.

Fusion, feedback, reranking

• Gordon Cormack, Charles Clarke, Stefan Büttcher (2009). *Reciprocal Rank Fusion Outperforms Condorcet and Individual Rank Learning Methods. SIGIR* ’09. The hybrid-search fusion baked into Elasticsearch, OpenSearch, Weaviate, and Azure. A constant set to 60 and never tuned, beating the trained methods. The detail I think about more than is healthy.
• Edward A. Fox, Joseph A. Shaw (1994). *Combination of Multiple Searches. TREC-2.* Combining the outputs of multiple retrieval systems, CombSUM and CombMNZ, already routine in 1994.
• Luyu Gao, Xueguang Ma, Jimmy Lin, Jamie Callan (2022). *Precise Zero-Shot Dense Retrieval without Relevance Labels (HyDE). arXiv / ACL 2023.* Hallucinate a fake answer, embed that instead of the question. Genuinely clever, and in spirit neural pseudo-relevance feedback.
• J.J. Rocchio (1971). *Relevance Feedback in Information Retrieval, in The SMART Retrieval System* (Gerard Salton, ed.). The original feedback move: nudge the query toward what looks relevant, away from what doesn’t. The 1971 in the subtitle.
• W. Bruce Croft, D.J. Harper (1979). Using Probabilistic Models of Document Retrieval without Relevance Information. Journal of Documentation 35(4). Pseudo-relevance feedback: assume your top results are good enough to learn from. They also named the failure, query drift, that bites every HyDE loop in production.
• Lidan Wang, Jimmy Lin, Donald Metzler (2011). *A Cascade Ranking Model for Efficient Ranked Retrieval. SIGIR* ’11. The formal version of your BM25-then-cross-encoder pipeline. A two-stage cascade, named, well before you shipped it.
• Chris Burges et al. (2005). *Learning to Rank using Gradient Descent (RankNet). ICML* ’05. Learning-to-rank had a working neural ranker in 2005. The reranker isn’t new; the transformer inside it is.

Evaluation

• Cyril Cleverdon, Jack Mills, Michael Keen (1966). *Factors Determining the Performance of Indexing Systems* (Cranfield II). The Cranfield paradigm: fix a corpus, write queries, collect judgments, score offline. Sixty years old. Your eval harness is a smaller, undocumented version of this.
• Ellen M. Voorhees, Donna K. Harman, eds. (2005). TREC: Experiment and Evaluation in Information Retrieval. MIT Press. The book on evaluating retrieval without fooling yourself. The unglamorous work behind a test set you can actually trust.
• Kalervo Järvelin, Jaana Kekäläinen (2002). *Cumulated Gain-Based Evaluation of IR Techniques. ACM TOIS* 20(4). Where nDCG comes from, and the justification behind it.
• Ellen M. Voorhees (2000). Variations in Relevance Judgments and the Measurement of Retrieval Effectiveness. Information Processing & Management 36(5). Human judges disagree, and she measured how much. The exact problem facing anyone who wants to swap a human judge for an LLM.
• Guglielmo Faggioli, Laura Dietz, Charles Clarke, et al. (2023). *Perspectives on Large Language Models for Relevance Judgment. ICTIR* ’23. The IR community arguing in print about whether you can let an LLM judge relevance. Where “named researchers warned it could corrupt the science” comes from.
• Bhaskar Mitra, Nick Craswell (2018). *An Introduction to Neural Information Retrieval. Foundations and Trends in IR* 13(1). Set the neural models inside the classical tradition, with BM25 as the baseline you have to respect. The map, drawn by people who knew the whole territory.
• Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees (2019). *Overview of the TREC 2019 Deep Learning Track*. They scored the neural era on the old field’s pooled, reusable test collections. It held up. The rigor was transferable the whole time.

The empirical results

• Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, Iryna Gurevych (2021). *BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models. NeurIPS* ’21. The receipt. In-domain performance doesn’t predict out-of-domain; BM25 beats most dense models the moment you leave the training distribution. Sitting there for four years before I read it.
• Jimmy Lin (2018). *The Neural Hype and Comparisons Against Weak Baselines. SIGIR Forum* 52(2). A lot of neural “wins” were wins over a BM25 nobody bothered to configure. Read it before you trust the next benchmark that shows vector search crushing “keyword search.”
• Wei Yang, Kuang Lu, Peilin Yang, Jimmy Lin (2019). *Critically Examining the ‘Neural Hype’: Weak Baselines and the Additivity of Effectiveness Gains from Neural Ranking Models. SIGIR* ’19. Went through years of papers on a standard collection and found no upward trend in effectiveness at all.
• Jimmy Lin (2019). *The Neural Hype, Justified! A Recantation. SIGIR Forum* 53(2). The same critic, better evidence, changing his mind in print. Half of the posture worth having.
• Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang (2023). *Lost in the Middle: How Language Models Use Long Contexts. TACL* 12 (2024). Models use information best at the start and end, worst in the middle. Adding retrieved context scored below the model’s own closed-book number.
• Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, Geri Gay (2005). *Accurately Interpreting Clickthrough Data as Implicit Feedback. SIGIR ’05, with the 2007 follow-up in ACM TOIS* 25(2). Attention pools at the top of a ranked list; users click the top even when something lower is better. A trust bias, eighteen years before we “found” it in models.
• Blerta Veseli, Julian Chibane, Mariya Toneva, Alexander Koller (2025). *Positional Biases Shift as Inputs Approach Context Window Limits. COLM* ’25. The dead zone drifts as inputs stretch toward the context limit, not reliably going away. Why “Lost in the Middle” is a moving target, not scripture.

What’s genuinely new

• Florin Cuconasu et al. (2024). *The Power of Noise: Redefining Retrieval for RAG Systems. SIGIR* ’24. Adding random, unrelated documents improved accuracy by as much as thirty-five percent. The atom of relevance changed, and most of our metrics still weigh the old one.
• Shahul Es, Jithin James, Luis Espinosa-Anke, Steven Schockaert (2023). *RAGAS: Automated Evaluation of Retrieval Augmented Generation*. Tooling to measure faithfulness: whether the generated text is actually supported by what you retrieved. A new axis the prior field never had to name, because its systems couldn’t lie.
• Omar Khattab et al. (2022, 2023). *Demonstrate-Search-Predict and [DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines](https://arxiv.org/abs/2310.03714)*. The serious version of agentic retrieval: decompose a question, fire your own sub-queries, re-plan. No accident it comes from someone who knows the old literature cold.
• *TREC 2024 RAG Track*, NIST. The establishment spinning up a track and inviting RAG inside. 301 topics, 113 million passages, nugget recall, citation support.

This story is published on Generative AI. Connect with us on LinkedIn and follow Zeniteq to stay in the loop with the latest AI stories.

Subscribe to our newsletter and YouTube channel to stay updated with the latest news and updates on generative AI. Let’s shape the future of AI together!