System Design Journey — Chapter 4: Latency vs. Throughput

Today’s lesson clarified one of the most common misconceptions in System Design: Latency and Throughput are not the same thing.

⏱️ Latency

Latency is the time taken to process a single request.

Example: How long does it take for a user’s profile to load after clicking it?

Lower latency = Better user experience.

🚀 Throughput

Throughput is the number of requests a system can handle in a given period of time.

Example: How many requests can the system process per second (RPS)?

Higher throughput = Better ability to handle traffic at scale.

A Simple Analogy

Think of a highway:

• Latency is how long one car takes to reach its destination.
• Throughput is how many cars pass through the highway every hour.

Both are important, but they measure completely different things.

One Key Learning

A system can have low latency but poor throughput, or high throughput with slightly higher latency.

For example, during a flash sale or concert ticket booking, millions of users may arrive simultaneously. In such scenarios, keeping the system operational (throughput) is often more important than ensuring every individual request is served with the lowest possible latency.

System design is all about making the right trade-offs based on the product’s requirements.

💡 Interview Tip

Whenever performance is discussed, ask yourself:

• Are we optimizing for one user’s experience? → Think Latency.
• Or are we optimizing for handling massive traffic? → Think Throughput.

Don’t just say, “The system should be fast.”

Instead, be specific:

• Latency → Measured in milliseconds (ms)
• Throughput → Measured in Requests Per Second (RPS) or Transactions Per Second (TPS)

Using the correct metric demonstrates that you’re thinking like a system designer, not just a software developer.