Embeddings and RAG explained to my parents

TL;DR: Last time, we saw why plain grep was no longer enough for my second brain, and why we needed to bring in RAG. Today, we’re going to demystify and clarify those two concepts — embeddings and RAG — to better understand what’s now going to happen inside my second brain.

📚 This article is part of the series “Second brain: RAG”. Each episode stands on its own; together they unpack key concepts of software engineering and AI, and they also tell the story of how I gave my second brain a semantic memory and made it better. At the end of this series, I’ll publish my second brain so you can build one for yourself.

***french version here***

👉 Haven’t read the previous episode yet? **Why my second brain was talking without understanding**.

group by meaning, not alphabetically.

Two scary words (for nothing)

In the previous episode, my second brain had a hole: it searched for exact words instead of meaning. A first name instead of a role, and it went blind. I promised a cure without defining it. Here it is, and it fits in two words: embedding and RAG.

On paper, it sounds like engineer jargon. In reality, it’s crystal clear. By the end of these 4 minutes, you’ll even be able to explain it to your parents too.

An embedding: grouping by meaning

Picture a huge room where you store everything you know. Most people would arrange things alphabetically, or by date. You arrange them by meaning: “holidays” right next to “beach” and “sun”, and very far from “tax return”.

An embedding is exactly that, but done by the machine. You give it a piece of text, and it assigns it a place in this space (in reality, a list of numbers — its coordinates). Two texts that talk about the same thing land right next to each other, even if they don’t share a single word. “Team workload” and “capacity management” become neighbors.

A small subtlety, because it matters: the machine doesn’t store a whole note in one block. It cuts it into small cards, and it’s each card that it places in the space. We’ll see the how in a future episode; for now, just remember that the unit being filed is the card, not the whole file.

The key point: the machine understands nothing. All it does is measure distances between points. But that’s enough to retrieve by meaning.

A RAG: search first, answer second

Now, RAG. Picture a consultant you ask a question. The consultant doesn’t know everything, but they can call a world-class expert whenever they need to.

Without RAG, the consultant passes your question straight to the expert, bare, with no context attached. The expert answers from memory. Being a generalist, the expert knows a bit of everything, but nothing precise about your files. If the expert knows, great. If not… they answer anyway, with confidence (this is called “hallucinating”).

With RAG, the consultant takes a detour before calling the expert. The consultant sends a library intern to fetch the few cards that genuinely speak to your question. The intern knows the shelves are arranged by meaning (thanks to embeddings prepared ahead of time on all your texts), so the intern quickly pulls the ones that count. The consultant then attaches those cards to the question for the expert: “here’s what we found, answer based on this.” The question is no longer bare: it arrives wrapped in the right context. Same expert, but this time the answer sticks to your documents.

In my second brain, the consultant is Claude Code on my machine, the intern is the search engine over my notes (my RAG MCP tools), and the expert is Anthropic’s model in the cloud.

That’s the whole difference: between an AI that improvises on its memories, and an AI that leans on your notes.

In a nutshell

• The embedding is the brick that files each card next to the others that share the same meaning.
• The RAG is the recipe that goes and fetches the right cards, then hands them to the LLM along with your question — so it can answer based on them.

One is inside the other: the embedding is part of RAG. No RAG without embeddings to retrieve the right cards; but the embedding, on its own, only files things: it’s RAG that uses it to answer.

Why it matters

This duo is what gave my second brain a semantic memory: ask a question in my own words, and retrieve the right notes even when they’re phrased differently, even when they’re scattered across several files.

Now it remains to actually build it.

Next episode: the architecture. Why I run this locally first, and send to the cloud only the bare minimum: the vectors. See you very soon!