Recommendation Systems

Going deep on Engineering blogs from Netflix , Meta , Google and Uber for Recommendation Systems.

One thing became clear quickly, they all independently built the same core architecture for Recommendation Systems and then diverged in exactly one critical decision each.

The universal pattern:

The universal pattern

Every company runs a two-stage architecture:

1. Retrieval: Retrieve 100M+ items, shortlist ~500 candidates in under 10ms using a Two-Tower neural net + approximate nearest neighbor index (HNSW or ScaNN).
2. Ranking: score all 500 with an expensive deep network in under 100ms.

That’s it. Netflix does it. Meta does it. YouTube does it. Uber does it. The insight is simple, you can’t run a billion-parameter model over 100M items per request. You need an inexpensive filter before an expensive scorer.

Where they diverge is where it gets interesting.

Meta built a DLRM, a model where the real bottleneck isn’t computation, it’s memory. Their embedding tables are terabytes in size. Random lookups across TB-scale tables crash CPU caches so badly they built a custom ASIC (ZION) just to handle embedding lookups. Not for matrix math. For memory access patterns.

Google built a Wide & Deep Learning bridges memorization (logistic regression over hand-crafted feature crosses) and generalization (deep neural net over embeddings) in a single jointly-trained model. They later replaced the wide side with a DCN cross network that learns feature interactions in O(d) per layer instead of O(d²). And they solved the stale model problem by moving to continuous streaming gradient updates. The model never stops training.

Netflix doesn’t optimize for clicks alone. So they built a multi-objective loss where the weights aren’t hand-tuned but learned via counterfactual evaluation using importance sampling reweighting. They also run a separate contextual bandit per title to personalize which document/series each user sees, updating hourly.

Uber doesn’t have a recommendation problem. They have a constrained matching problem under hard real-time deadlines. The objective is maximize trip value subject to driver ETA, limits, surge price bounds, and fairness constraints all solved in under 100ms using gradient boosted trees for ETA + deep RL for pricing + bipartite matching for final assignment.

Architecture decision comparison

End to end latency serving per request

The single biggest lesson across all four:The architectural fix is always the sam, add long-horizon reward signals to your loss function. But doing that requires years of logged data, causal modeling infrastructure, and the organizational willingness to accept short-term metric drops for long-term retention.The hardest part of building a recommendation system isn’t the Two-Tower model or the ANN index. It’s deciding what you’re actually optimizing for.