RAG: Why and what are Embeddings , How Embedding stored in Vector DB

What are embeddings?

An embedding is a way of turning a piece of text (a word, sentence, paragraph, whatever) into a list of numbers — a vector.

for example “The cat sat on the mat” → [0.12, -0.45, 0.89, …, 0.03] (usually 384–3072 numbers)

These numbers aren’t random. They’re produced by a neural network trained so that texts with similar meaning end up close together in this numerical space, and texts with different meaning end up far apart.

Texts with similar meaning are placed near each other, so “car” and “automobile” end up close even if they do not share the same keyword

Why not just keyword search or Why embedding is essential ?

Keyword search matches exact words or close variants. That is fast, but it fails when the query and document use different wording for the same idea.

For example, a user may search “how to end a subscription,” while the document says “cancel membership” — keyword search may miss that, but embeddings can still match the meaning.

Problem with keyword search : Keyword search has no concept of meaning — it only knows string overlap. It fails at:

• Synonyms (“purchase” vs “buy”)
• Paraphrasing (different sentence structure, same meaning)
• Context/disambiguation (“Apple” the company vs the fruit)
• Cross-lingual or misspelled queries

Solution : Embeddings solve this because they capture semantic meaning, not just surface text. So in a RAG (Retrieval-Augmented Generation) system, when a user asks a question, you convert the question into an embedding, then search your document chunks (which were pre-embedded) for the ones whose vectors are closest — even if they don’t share a single keyword. That’s why it’s called semantic search.

Cosine similarity

Cosine similarity measures how close two vectors are by comparing the angle between them. In simple terms, it tells you whether two embeddings point in a similar direction, which is why it is commonly used in semantic search and vector databases.

Once you have two vectors, how do you measure “closeness”? The most common method is cosine similarity.

Instead of measuring the straight-line distance between two vectors, cosine similarity measures the angle between them:

• If two vectors point in exactly the same direction → angle = 0° → cosine similarity = 1 (identical meaning)
• If they’re perpendicular → cosine similarity = 0 (unrelated)
• If they point in opposite directions → cosine similarity = -1 (opposite meaning)

Putting it together in a RAG pipeline

1. Indexing time: Split documents into chunks → embed each chunk → store vectors in a vector database (e.g., Pinecone, Weaviate, FAISS, pgvector)
2. Query time: Embed the user’s question with the same embedding model
3. Retrieval: Compute cosine similarity between the query vector and all stored chunk vectors → return the top-k most similar chunks
4. Generation: Feed those retrieved chunks + the user’s question into the LLM as context, so it can generate a grounded answer

How are embeddings stored in vector databases for RAG ?

Embeddings are stored in a vector database as vector records, usually one record per chunk of text. Each record typically includes the embedding vector itself, plus metadata like document ID, source, page number, chunk index, and sometimes the original text chunk for display or re-ranking.

How Vector DB works ?

1. You split the document into chunks.
2. You convert each chunk into an embedding vector.
3. You insert that vector into the vector database with metadata.
4. The database builds an index so it can do fast similarity search later

A typical vector DB entry looks like this:

Example

If a document is split into three chunks, the vector DB may store three separate records like this:

• Chunk 1 → embedding + {doc_id, page: 1, chunk: 1}
• Chunk 2 → embedding + {doc_id, page: 1, chunk: 2}
• Chunk 3 → embedding + {doc_id, page: 2, chunk: 3}

When a query comes in, the query is also embedded, and the database returns the stored vectors that are most similar to it.

Vector DB — Which One Should You Use?

1. ChromaDB — Open-source, developer-friendly, great for RAG prototypes and local apps.
2. Pinecone — Fully managed, serverless, built for enterprise-scale, fast & reliable.
3. Weaviate — Hybrid search (text + vector), schema support, great for multimodal data.
4. FAISS — Fast, offline/local, customizable, ideal for research & on-device use.
5. Milvus — Scalable and GPU-accelerated, production-ready with high concurrency.
6. Qdrant — RESTful API, Rust-based, great performance and docker-friendly.
7. Redis (with Vector Support) — For teams already using Redis, now supports vector similarity!