What Is Reciprocal Rank Fusion and Why I Used It in My RAG System

When you build a system that retrieves information before generating an answer, your first real problem is not the generation — it is the retrieval. Specifically, it is this: how do you decide which pieces of information are actually the most relevant ones to pass to the model?

The naive answer is to search for them. But search is not one thing. There are at least two meaningfully different ways to find relevant content, and they disagree with each other more often than you would expect.

Keyword search — the kind that looks for exact term matches — is good at finding documents that contain the precise words someone typed. Ask it for “BM25 retrieval” and it will find every document that contains those words. What it cannot do is understand that “finding relevant documents using term frequency” is describing the same concept. It is fast and precise, but it is also literal in a way that causes it to miss meaning.

Semantic search works differently. It converts text into numerical representations that capture meaning, and finds documents that are conceptually close to the query even when the words do not match exactly. It understands that “car” and “automobile” are related. What it sometimes misses is the exact terminology — if someone searches for a specific function name or a product code, semantic similarity is less useful than a direct keyword hit.

Neither method is wrong. They are looking for different things. Which is why, if you are building a retrieval system that needs to be reliably useful, you eventually want both running at the same time. That decision creates a new problem.

Running two retrieval methods in parallel means you end up with two separate ranked lists of results. Each list contains documents ordered by how relevant that method thinks they are. The lists overlap — the same document might appear in both — but they rarely agree on the ordering. A document that ranks third in the keyword list might rank twelfth in the semantic list. Another might be first semantically and not appear in the keyword results at all.

You have two lists. You need one. How do you combine them?

The instinct is to average the scores. Take the score each method assigned to each document, average them, and re-rank by the combined number. The problem is that scores from different retrieval methods are not on the same scale. A BM25 score of 12 and a vector similarity score of 0.87 are not comparable numbers — they are measuring fundamentally different things in different units. Averaging them produces a combined score that is mathematically meaningless. You would be adding feet to kilograms.

This is the specific problem that Reciprocal Rank Fusion solves.

RRF sidesteps the score comparison problem entirely by ignoring scores and working with ranks instead. Position in the list is something both methods share, regardless of how they calculate relevance internally.

The mechanism is straightforward. For each document, in each ranked list it appears in, you calculate a reciprocal rank score using its position. A document ranked first gets a higher score than one ranked fifth, which gets a higher score than one ranked twentieth. The exact formula includes a small constant — typically 60 — added to the rank before taking the reciprocal, which prevents the top-ranked documents from being weighted so heavily that everything else becomes irrelevant. Then you add the reciprocal rank scores from each list together to get a final combined score, and re-rank by that.

A worked example makes this concrete. Imagine three documents — call them A, B, and C.

In the keyword search list, Document A ranks first, Document B ranks second, Document C ranks third. In the semantic search list, Document C ranks first, Document B ranks second, Document A ranks fifth.

If you combined by scores, the result would depend entirely on the arbitrary scale of each method. But with RRF, Document B — which ranked second in both lists — ends up with the highest combined score, even though it was never first in either. Document C, which topped the semantic list but ranked third in keyword search, comes second. Document A, which led the keyword list but ranked fifth semantically, ends up third.

The document that ranked consistently well across both methods won. That is the entire idea. RRF rewards broad relevance over narrow dominance.

For a RAG system specifically, this matters more than it might seem. The quality of what the model generates is almost entirely determined by the quality of what gets retrieved. Pass it the wrong chunks and it will either hallucinate, produce vague answers, or confidently say something that the source material does not actually support. Retrieval is not a preprocessing step — it is the foundation.

RRF makes that foundation more robust without requiring you to manually tune how much weight to give each retrieval method. There are no coefficients to calibrate, no hyperparameters to search over. You run both methods, apply the fusion formula, and the ranking that emerges reflects genuine consistency across both signals.

In the RAG system I built, the retrieval pipeline ran BM25 and vector search in parallel — BM25 for exact keyword matching against the document store, vector search for semantic similarity using embeddings. Both methods returned their own ranked lists of document chunks. Those lists were merged using RRF, which produced a single re-ranked set of results. The top chunks from that merged list were passed to the language model to generate the final grounded answer.

The honest thing to add: I understand the mechanism well enough to have implemented it and to explain why it works. What I am still less certain about is how to evaluate whether my specific constant value and chunk size are actually optimal for the type of documents I was retrieving from. That is the part I am still working through.

RRF is not a sophisticated solution. It is a clean one. It solves a specific, well-defined problem — how to combine ranked lists from incompatible scoring systems — without introducing new complexity in the process. If you are building your first RAG system and you have decided to use hybrid search, RRF is the right place to start before reaching for learned reranking models or more complex fusion approaches. It will almost certainly outperform either retrieval method alone, and it will do it without requiring you to tune anything.

If you have run into retrieval quality problems in your own projects — results that felt slightly off, answers that missed the point despite the documents being there — what did you try? I am curious whether others have found RRF sufficient or hit the ceiling of it quickly.