What Happens in the 5 Milliseconds After You Hit “Search” on Medium?

The Day My Brain Turned into a Search Engine

So, there I was the other day, scrolling through Medium (you know, that massive website where thousands of people write stories every single day). I was looking for a very specific article about a rare dog breed, so I typed “Golden Retriever puppy training tricks” into the search bar.

Bam. In less than a blink of an eye — maybe 5 milliseconds — the exact articles I needed popped up on my screen.

Normally, I wouldn’t think twice about it. But lately, I’ve been studying Data Structures and Algorithms (DSA) for hours on end. If you’ve ever played Tetris for too long, you know how you start seeing falling blocks everywhere you look? That’s me, but with code. The “Tetris Effect” has completely taken over my brain. Every time I organize my clothes, I think of Sorting Algorithms. When I look at a family tree, I see a Binary Tree.

So when Medium gave me those search results instantly, my brain didn’t just say “cool.” It panicked. I thought: “Wait a minute. Medium has millions of articles. If a computer had to read every single word of every single article to find the word ‘Retriever,’ it would take forever! That’s an $O(N)$ linear time operation! How is this so fast?”

I couldn’t sleep. I had to look under the hood. And that is how I discovered the absolute magic of Elasticsearch — the super-engine running behind the scenes of websites like Medium.

Here is exactly how this wizardry works, explained simply.

1. The Magic Textbook (The Inverted Index)

In a normal database, the computer keeps a list of articles, and each article contains a bunch of text. To find a word, the computer has to flip through every page of every book.

Elasticsearch does the exact opposite. It uses a data structure called an Inverted Index.

Imagine you are reading a giant history textbook. If you want to find where it talks about “Julius Caesar,” you don’t read the whole book from page one. You flip to the very back of the book to the Index, find the letter ‘C’, look up “Caesar,” and it tells you exactly: Pages 45, 89, and 120.

Elasticsearch builds that index before you even search. When an author publishes a blog post on Medium, Elasticsearch immediately breaks the blog into individual words. Instead of the article owning the words, the words own the article.

It creates a giant dictionary. The word “Retriever” points directly to a list (called a Postings List) containing: [Article #42, Article #1005, Article #8832]. When I searched for it, Elasticsearch didn't read any articles; it just looked at the back of its magic textbook, grabbed that list, and handed it to me.

2. The Word-Cleaning Factory (The Analyzer Pipeline)

But wait, human language is messy. What if the article said “Retrievers” (plural), but I searched for “Retriever” (singular)? Or what if I capitalized the words?

Before any word gets written into that magic textbook index, Elasticsearch sends it through a factory conveyor belt called the Analyzer Pipeline.

• Stage 1: The Cleaner (Character Filters): It scrubs out ugly HTML tags or weird symbols.
• Stage 2: The Chopper (Tokenizer): It takes a sentence like “The quick brown fox” and chops it up into neat, individual word blocks: ["The", "quick", "brown", "fox"].
• Stage 3: The Transformer (Token Filters): This is the coolest part. It turns all words into lowercase so capital letters don’t confuse the system. Then, it uses a tool called a Stemmer to chop off word endings. It turns “running,” “runs,” and “ran” all into just “run.” It turns “puppies” and “puppy” into just “puppi.”

Because it runs this exact same factory on both the articles and your search query, searching for “Puppies” matches an article written about a “puppy” perfectly!

3. Storage Blocks That Never Change (Immutable Segments)

As a DSA student, I know that changing data in a database can cause a lot of traffic jams (locks). Elasticsearch avoids this with a clever trick: Immutability.

When it writes these word lists to the computer’s hard drive, it writes them in small, frozen chunks called Segments. Once a segment is written, it can never be altered.

Because these blocks never change, the computer doesn’t have to worry about data getting messy. It can safely copy them straight into the computer’s super-fast short-term memory (RAM). Reading from memory is infinitely faster than reading from a spinning hard drive, which is why the results fly back to you in milliseconds.

(Bonus DSA Trivia: If you delete a post, Elasticsearch doesn’t actually erase it right away because the block is frozen! It just puts a “Tombstone” sticker on it. During a quiet time later, it merges old blocks together and throws away the stickered files.)

4. The Sorting Judge (BM25 Relevance Scoring)

If my search for “Golden Retriever” matches 10,000 articles on Medium, how did it know which one to put at the very top of my page? It uses a mathematical judge called the BM25 Algorithm. It grades articles based on three smart rules:

1. Term Frequency: If Article A mentions “Retriever” 15 times, and Article B mentions it once, Article A is probably a better match. (Though if it mentions it 100 times, the judge stops giving extra points — that’s called saturation!).
2. Length Normalization: If a short 2-paragraph tweet mentions “Retriever,” it’s definitely about dogs. If a 600-page encyclopedia mentions “Retriever” once, it’s probably just a passing comment. Shorter fields get higher scores!
3. Inverse Document Frequency (IDF): Words like “the,” “and,” or “training” are everywhere. They aren’t special. But a word like “Retriever” is relatively rare. The judge gives way more points for matching rare, unique words than common ones.

5. Teamwork Makes the Dream Work (The Distributed Layer)

Finally, Medium has way too much data to fit on one computer. Elasticsearch handles this by splitting the giant magic textbook into smaller chapters called Shards, and scattering them across a whole team of computers called Nodes.

When I type my search query, it hits one computer called the Coordinating Node. This computer acts like a project manager:

• The Scatter: It screams out to all the other computers at the exact same time: “Hey! Check your chapters and give me your top 10 best matching articles!”
• The Gather: The other computers do their local search and send back just the IDs and scores of their top 10 choices. The manager merges all the lists together, picks the ultimate top 10 winners, and fetches the actual full articles to show on my screen.

The Tetris Blocks Fall into Place

And that’s it! By moving all the heavy mathematical lifting to the moment the article is written, rather than the moment it is searched, Elasticsearch completely hacks the time complexity rules that were keeping me up at night.

So the next time you search for something online and get an answer instantly, remember: you aren’t watching a computer read really fast. You’re watching the beautiful, coordinated dance of an Inverted Index, a word factory, and a highly organized team of computers working together in perfect harmony.

My DSA brain can finally rest easy tonight

**Bonus: **Visuals and architectural insights used in this article are adapted from the excellent breakdown “Why Elasticsearch Is So Fast” by PawelCodeStuff on YouTube.