103 Vespa | Inverted Index

How Does Vespa Find a Document in Milliseconds Without Reading Millions of Them?

Imagine you’re running an e-commerce site with 100 million products.

A user types:

wireless noise cancelling headphones

and expects results almost instantly.

Now think about what the search engine could do.

One option is painfully obvious:

Open product #1
Does it contain "wireless"?
Does it contain "noise"?
Does it contain "headphones"?

Open product #2
Repeat.

Open product #3
Repeat.

Do this 100 million times and your users might get their results sometime next week.

Yet Vespa returns results in milliseconds.

How?

That’s the mystery.

The Librarian Trick

Imagine a giant library.

A visitor walks in and asks:

“Show me every book that mentions Vespa.”

The librarian has two choices.

Option 1: Read Every Book

Clearly impossible.

Option 2: Keep a Special Catalog

Instead of listing books by title, the catalog lists words.

vespa        → Book 12, Book 45, Book 77
architecture → Book 12, Book 77
search       → Book 45, Book 77, Book 91

Now when someone asks for “vespa”, the librarian doesn’t search the books.

The answer is already waiting:

Book 12
Book 45
Book 77

The search becomes a lookup instead of an investigation.

This simple idea is what powers modern search engines.

And inside Vespa, it lives in a structure called the inverted index.

Where Does This Catalog Live?

When people first learn Vespa, they often imagine a huge search index sitting somewhere on disk.

The reality is more interesting.

The diagram below shows where the inverted index lives inside Vespa’s architecture.

At the center is the Content Cluster, powered by Proton.

Think of Proton as the component responsible for storing documents and making them searchable.

Inside Proton, several storage structures work together:

• Inverted Index
• Attribute Store
• Document Store

Each has a different job.

A Document Arrives

Let’s say we feed a document into Vespa:

{
  "id": "doc:12",
  "title": "Vespa architecture search engine"
}

The document enters the Content Cluster.

At this point Vespa faces another challenge.

Humans search using words.

Computers store bytes.

Some translation must happen.

Breaking Text Into Searchable Pieces

Before anything can be searched, Vespa analyzes the text.

The title:

Vespa architecture search engine

becomes:

vespa
architecture
search
engine

These individual pieces are called tokens.

Now Vespa knows the important searchable terms contained in the document.

But Where Does Vespa Put Them?

A common assumption is:

“It immediately writes everything to disk.”

That would be safe.

It would also be slow.

Imagine every incoming document waiting for a disk write before becoming searchable.

At scale, ingestion throughput would collapse.

So Vespa takes a different path.

The First Stop: Memory

The tokens are first placed into an in-memory structure called the MemIndex.

The MemIndex might now contain:

vespa         → [12]
architecture  → [12]
search        → [12]
engine        → [12]

Something remarkable has already happened.

The document is searchable.

Even though nothing has been flushed to disk yet.

This is how Vespa achieves near real-time indexing.

The Growing Notebook Problem

Over time, more documents arrive.

The in-memory index grows.

Eventually it becomes too large to keep expanding forever.

Vespa needs a permanent version.

So periodically it performs a flush.

From Memory to Durable Storage

The MemIndex is written to disk as immutable segments.

Conceptually:

Memory
   ↓
segment_1.idx
segment_2.idx
segment_3.idx

Now the data is durable.

If a node restarts, the index survives.

The interesting part is that searches can continue while new documents are being indexed.

The system balances:

• Fast writes in memory
• Durable storage on disk
• Fast lookups during search

all at the same time.

A User Finally Searches

Hours later, someone types:

vespa

What happens next?

Many people imagine Vespa opening every document and checking whether the word appears.

It doesn’t.

Not even close.

Instead, Vespa performs one lookup.

vespa → [12, 45, 77]

In a fraction of a millisecond, it discovers every matching document.

The hard work was already done during indexing.

Search simply reuses that preparation.

Finding Matches Is Only Half the Story

Suppose the lookup returns:

[12, 45, 77]

Which document should appear first?

All three contain the word.

But users expect relevance.

This is where the rest of Proton enters the picture.

The Other Stores Join the Search

The inverted index only knows:

term → document IDs

It does not store the actual document content.

For that, Vespa consults the Document Store.

Need field values for ranking, filtering, or sorting?

Vespa consults the Attribute Store.

Together they provide the information required to score the candidates.

Ranking the Results

Now Vespa can evaluate:

• How often does the term appear?
• Is it in the title?
• Is it in the body?
• How important is the field?
• What ranking profile is configured?

A ranking algorithm such as BM25 assigns scores.

For example:

doc:12   score 1.72
doc:77   score 1.36
doc:45   score 0.98

The highest score wins.

The user receives results ordered by relevance instead of randomness.

The Bigger Picture

Looking back at the diagram, notice the separation of responsibilities. The Container Cluster receives and processes queries. The Content Cluster stores documents and executes searches. The Config Cluster distributes configuration and keeps the system coordinated.

This separation allows Vespa to scale independently:

• More queries → scale containers
• More data → scale content nodes
• More configuration management → handled centrally

The Real Secret

The reason Vespa can search millions or billions of documents so quickly isn’t that it searches faster.

It’s that it avoids searching altogether.

The expensive work happens when documents are indexed.

By the time a user types a query, Vespa already knows exactly where every term lives.

The search request becomes:

Find "vespa"
        ↓
Lookup index
        ↓
Get matching IDs
        ↓
Fetch metadata
        ↓
      Rank
        ↓
Return results

What looks like magic to the user is really a carefully prepared shortcut built long before the query ever arrived.

Hawkeye Summary