From Embeddings to Search: FAISS, HNSW, and IVF-PQ Made Simple for Engineers

How engineers build millisecond-scale similarity search across millions of embeddings

You need to search through 100 million embeddings in milliseconds. Traditional databases crumble; scanning every record would take minutes.

Enter FAISS, Meta’s open-source vector search engine. It makes semantic search, recommendations, and RAG systems lightning fast by using Approximate Nearest Neighbour (ANN) algorithms like HNSW and IVF-PQ.

This post walks you through everything from basic concepts to production-grade optimization, with architecture visuals, tuning guides, benchmark data, and real-world case studies.

What Are Vector Databases?

When you represent text, images, or users as embeddings, i.e., vectors, you can measure how similar two things are by comparing their numeric distance.

But what if you have 100M vectors and need the closest 10 to a query?

Traditional databases aren’t built for this fuzzy, high-dimensional math. That’s where vector databases (and engines like FAISS) come in. They specialize in storing and searching vectors efficiently.

FAISS Overview

FAISS (Facebook AI Similarity Search) is a C++/Python library optimized for fast, large-scale similarity search. It provides a range of index types from exact search (IndexFlatL2) to advanced ANN structures like HNSW and IVF-PQ.

Why FAISS?

• GPU acceleration (10–100x faster than CPU)
• Memory-efficient compression via quantization
• Tunable trade-offs: accuracy vs latency vs memory
• Extensible for custom similarity metrics

FAISS System Architecture

Foundational Concepts

Index Types in FAISS

1. IndexFlatL2 / IndexFlatIP Exact Search

• Scans all vectors → 100% accuracy
• Memory-heavy and slow beyond ~1M vectors
• Use case: small datasets or recall benchmarking

2. HNSW Hierarchical Navigable Small World

A graph-based ANN structure. Each vector is a node connected to a few neighbours; higher layers link distant clusters, lower layers link close ones.

Analogy: Imagine finding your way across a city using highway exits first (top layer), then smaller roads (lower layers).

Parameters to tune:

• M: number of connections per node
• efConstruction: how widely to search during build
• efSearch: how widely to explore during query

3. IVF-PQ Inverted File + Product Quantization

A two-stage approach:

1. IVF (Inverted File): Group vectors into nlist clusters via K-Means. Search only in the nearest clusters (nprobe).
2. PQ (Product Quantization): Compress vectors into short binary codes to save memory.

Analogy: Like sorting books into genre rooms (IVF) and storing short summaries (PQ) to save space.

Key Parameters:

FAISS in Code

Setup & Flat Index

import faiss, numpy as np
d = 128
xb = np.random.randn(100_000, d).astype('float32')
index = faiss.IndexFlatL2(d)
index.add(xb)
xq = np.random.randn(5, d).astype('float32')
D, I = index.search(xq, 5)

HNSW Example

index = faiss.IndexHNSWFlat(d, M=32)
index.hnsw.efConstruction = 200
index.hnsw.efSearch = 64
index.add(xb)
D, I = index.search(xq, 10)

IVF-PQ Example

quantizer = faiss.IndexFlatL2(d)
index = faiss.IndexIVFPQ(quantizer, d, 4096, 32, 8)
index.train(xb)
index.add(xb)
index.nprobe = 16
D, I = index.search(xq, 10)

Parameter Tuning Cheatsheet

Index Selection Guide

Cost Analysis

Benchmarking Protocol

• Datasets: SIFT1M, DEEP1B, custom embeddings
• Metrics: Recall@10, Recall@100, Latency (p50/p95), Memory, Build Time
• Tools: ann-benchmarks, FAISS profiler, Python scripts
• Procedure:
• Create ground truth via IndexFlatL2 on a subset
• Evaluate HNSW & IVF-PQ across parameter grids
• Plot recall vs latency and recall vs QPS

Troubleshooting Common FAISS Issues

Vector DB Landscape (Beyond FAISS)

Monitoring & Production Optimization

Dashboard Concept:

• Panels for:
• QPS / latency percentiles
• Recall@k over time
• Index rebuild metrics
• Memory / GPU utilization
• Threshold alerts: recall drift, slow queries, imbalance in IVF clusters

References & Resources

• FAISS Documentation
• FAISS GitHub Repository
• ANN Benchmarks
• Milvus Documentation
• Pinecone vs FAISS: Vector Search Comparison

Conclusion

FAISS is not just a library; it’s the backbone of semantic search, RAG systems, and large-scale recommendation engines. Once you understand HNSW and IVF-PQ, you can tailor speed, accuracy, and memory to your exact workload.