Hybrid Search: Combining the Best of Keyword and Semantic search

In the previous article, we covered the foundational building blocks of Search: ‘what are embeddings’, ‘how does text get converted into vectors’ and ‘how vector databases store those vectors’. If you haven’t read it, I would recommend starting there before continuing.

In this article, we’re going to build on that foundation and look at the retrieval side of things. When it comes to modern information retrieval, the two main approaches are Keyword search and Semantic search — with modern vector databases increasingly combining both into a single Hybrid search pipeline.

Hybrid Search

Let’s go through both of these approaches,

Keyword Search: The Traditional Approach

Keyword search works largely by matching the exact words in a query with those in your documents or database. So if you search for “best pizza restaurants”, keyword search will look for documents with those exact words.

So how does Keyword Indexing work? To get the best results with keyword search, we need to perform a series of text processing steps before we index our documents. Here’s a simplified view of the process,

Text processing and Indexing

Let us understand each one in a simple way,

1. Tokenization: The document text is split into individual tokens (words). For example, “Running shoes for trail hiking” becomes [“running”, “shoes”, “for”, “trail”, “hiking”]
2. Normalization: Tokens are lowercased and common words (stop words) like ‘for’, ‘the’, ‘and’ are removed, as they don’t carry much meaning for the search
3. Lemmatization: Using this technique, words are reduced to their root form, ‘running’ -> ‘run’, ‘hiking’ -> ‘hike’. This is done so that different forms of the same word are treated in a similar way. I have seen this is especially useful when dealing with multi-lingual text, where some languages have rich morphology (e.g. German, Russian, Swedish etc.) where a single word can have many different forms
4. Inverted Index construction: Once the pre-processing steps are done, the processed tokens need to be indexed in a way that allows fast retrieval. This is typically done using an ‘inverted index’ strategy, which maps each token to the documents that contain it. So for our example the inverted index would have entries like,

Term        → Documents (with term frequency)
──────────────────────────────────────────────
"running"   → [doc_1 (tf:2), doc_2 (tf:1)]
"shoes"     → [doc_1 (tf:1), doc_3 (tf:3)]
"trail"     → [doc_1 (tf:1)]
"hiking"    → [doc_1 (tf:2), doc_4 (tf:1)]

Keyword Search (Retrieval) Now that we have our inverted index created, lets see how the retrieval process works. When a user enters a search query, the search engine processes the query in the same way as we did for the documents (tokenization + normalization + lemmatization). Then it looks up each token in the inverted index to find matching documents. The engine then uses scoring algorithms to rank the results based on how relevant they are to the query. Common scoring algorithms include TF-IDF and BM25.

TF-IDF (Term Frequency-Inverse Document Frequency) is a common scoring algorithm that gives higher scores to documents that contain the query terms more frequently, while also considering how common those terms are across all documents.

BM25 (Best Matching 25) is an improvement over TF-IDF that takes into account the length of the document and the average document length in the corpus, providing a more balanced relevance score.

Keyword search is fast and efficient for exact matches, but it can struggle with synonyms, misspellings, and understanding the context of the query. For example, if a user searches for “best hiking shoes”, keyword search might not return documents that use the term “trail shoes” even though they are relevant. This is where Semantic search comes into play.

Semantic Search: Understanding the Meaning

Semantic search goes beyond just matching keywords. It aims to understand the meaning behind the query and the documents. This is achieved through the use of embeddings, which we covered in the first article. By converting text into vectors, semantic search can capture the underlying meaning and context of the words, allowing it to find relevant documents even if they don’t contain the exact keywords.

For example, if a user searches for “best hiking shoes”, semantic search can return documents that talk about “trail shoes” or “outdoor footwear” because it understands the semantic relationship between these terms. This makes semantic search particularly powerful for handling synonyms, misspellings and more complex queries that require an understanding of context.

The pre-processing steps for semantic search pretty much remain the same to that of a Keyword search (tokenization, normalization, lemmatization), but instead of creating an inverted index, we generate embeddings for the documents and store them in a vector database.

During retrieval, the query is also converted into an embedding and the search engine performs a similarity search to find the most relevant documents based on their vector representations.

Vector Indexing Strategy So how does the vector database find the nearest vectors quickly when we have millions of entries? Lets first understand the concept of distance in a vector space. When we convert text into vectors, we can measure how similar two pieces of text are by calculating the distance between their corresponding vectors. Remember the diagram we saw in the first article where we had a 2D vector space with points representing different texts?

The closer the vectors are in the vector space, the more similar the texts are in meaning. There are different ways to measure this distance, such as cosine similarity, dot product or Euclidean distance. The choice of distance metric can affect the relevance of search results. Here is a nice article that explains them if you want to read more.

When we have a large number of vectors, we need an efficient way to find the nearest neighbors (the most similar vectors) without having to compare the query vector with every single vector in the database. This is where different indexing strategies come into play. The two main strategies are:

1) Exact Nearest Neighbor (kNN) kNN is a brute-force method. When you submit a query, the database calculates the exact distance (using metrics like Cosine Similarity or Euclidean Distance) between your query vector and every single vector in the database. Gives exact results but becomes slow and expensive at scale. This technique is useful for small datasets or offline evaluation

2) Approximate Nearest Neighbor (ANN) ANN trades a tiny bit of accuracy for massive gains in speed. Instead of checking every single vector, ANN uses specialized data structures (indexes) to narrow down the search space to a small neighborhood of likely candidates. Returns very high-quality “near exact” results with much lower latency. This is the default strategy in most production systems. Some popular ANN index types are HNSW (Hierarchical Navigable Small World), IVF(Inverted File Index), PQ/IVF-PQ (Product Quantization).

Practical recommendation: - For text semantic search, start with HNSW + cosine similarity

• If vectors are normalized by your model or database pipeline, dot product is also a strong choice
• Move to IVF or IVF-PQ when dataset size and cost constraints become dominant

The Fusion: Hybrid Approach

While both Keyword and Semantic search have their strengths and weaknesses, combining them can provide the best of both worlds. This is known as hybrid search and is a common approach in most modern search engines. The key insight is that both searches run independently and in parallel on the full document corpus. For example, if a user searches for “best hiking shoes”, the keyword search will find documents containing those exact words, while the semantic search simultaneously finds documents about “trail footwear” or “outdoor running gear”. By merging both ranked lists, the final results benefit from both precision and contextual breadth.

Hybrid Search pipeline

Normalization of scores is an important step in the fusion process. Since keyword and semantic search may produce scores on different scales(say 0–1 or 1–10) we need to normalize them to a common range before merging. This ensures that neither search approach dominates the final ranking unfairly.

The results are then merged using a fusion strategy like Reciprocal Rank Fusion(RRF). In simple terms, RRF combines the ranked lists from both searches by giving higher weight to documents that appear in the top ranks of either list. This way, documents that are highly relevant in either search method are more likely to appear at the top of the final results.

You can throw in an optional re-ranking step after the fusion to further refine the results. This can be done using a more sophisticated model that takes into account additional features such as user behavior, document metadata or even a machine learning model trained to predict relevance. In the end, we achieve a more robust and effective search experience that leverages the strengths of both keyword and semantic search, providing users with more relevant and comprehensive results.

In our next article, we will use this Hybrid search as our RAG (Retrieval-Augmented Generation) pipeline to build a question answering system which is our last piece of the puzzle in our journey to build a modern search engine. So stay tuned!