How to Avoid Cache Stampede on Prod?

This is one of those situations where the problem isn’t the tool.

The problem is how we use the tool.

What Is a Cache Stampede?

A cache stampede occurs when many requests simultaneously encounter a cache miss for the same popular key and all try to regenerate that value from the underlying database or service.

Consider a popular endpoint:

GET /products/123

The application normally works like this:

Request
   ↓
Check Cache
   ↓
Cache Hit
   ↓
Return Data

This protects the database because most requests never reach it.

But what happens when the cache expires?

Request 1 ──┐
Request 2 ──┤
Request 3 ──┤
Request 4 ──┤
Request 5 ──┤
            ↓
        Cache Miss
            ↓
       Database

If hundreds or thousands of requests arrive at approximately the same time, they may all query the database to regenerate the same value.

Instead of:

1000 requests
      ↓
1000 cache hits
      ↓
       DB

you suddenly get:

1000 requests
      ↓
1000 cache misses
      ↓
1000 database queries
      ↓
Database overload

The problem becomes particularly severe when the key represents a hot resource that receives a large percentage of your traffic.

Why Is Cache Stampede Dangerous?

The database usually isn’t prepared to perform the same expensive operation hundreds or thousands of times simultaneously.

For example, imagine an endpoint that takes 500 ms to generate its response because it performs:

• multiple database queries
• joins
• aggregation
• expensive calculations

Normally, the result is cached, so subsequent requests are cheap.

But when that cache expires:

Cache expires
     ↓
500 requests arrive
     ↓
500 expensive DB operations
     ↓
DB connection pool fills
     ↓
Queries become slower
     ↓
More requests pile up
     ↓
Application latency increases

This can create a feedback loop where a single cache expiration eventually affects the entire application.

Cache Stampede vs Cache Avalanche

These two concepts are related but shouldn’t be treated as the same problem.

Cache Stampede

A stampede usually revolves around a hot cache key.

For example:

product:123

expires.

Thousands of users request product:123 at roughly the same time.

product:123
                  ↓
               EXPIRED
                  ↓
       ┌──────────┼──────────┐
       ↓          ↓          ↓
    Request 1  Request 2  Request 3
       ↓          ↓          ↓
      DB         DB         DB

The key problem is simultaneous regeneration of the same cached value.

Cache Avalanche

A cache avalanche happens when many cache entries expire or become unavailable around the same time, producing a large number of cache misses across the application.

For example:

user:123       → expires
product:456    → expires
dashboard:789  → expires
settings:111   → expires
report:222     → expires
       ↓
Large number of cache misses
       ↓
Large increase in DB traffic
       ↓
Database overload

This can happen when many keys are created around the same time with identical TTLs.

For example:

Key A → TTL 3600 seconds
Key B → TTL 3600 seconds
Key C → TTL 3600 seconds
Key D → TTL 3600 seconds

If these keys are populated together, they can also expire together.

Cache Penetration: Another Problem to Know

There is another cache-related problem worth understanding: cache penetration.

Cache penetration happens when requests repeatedly ask for data that doesn’t exist.

For example:

GET /users/999999
        ↓
Cache miss
        ↓
Database
        ↓
User doesn't exist

If the application doesn’t cache that negative result, the next request does the same thing.

An attacker or buggy client could repeatedly request nonexistent IDs:

/users/999999
/users/999998
/users/999997
/users/999996
...

The cache cannot help because there is no value to cache.

Common mitigations include:

• negative caching
• Bloom filters
• request validation
• rate limiting

So, at a high level:

ProblemWhat happensCache StampedeMany requests regenerate the same expired keyCache AvalancheMany cache keys expire around the same timeCache PenetrationRequests repeatedly query data that doesn’t exist

How to Prevent Cache Stampede

There isn’t one universal solution. The right approach depends on how expensive the underlying operation is and how frequently the data changes.

1. Use a Lock for Cache Regeneration

One of the most common approaches is to allow only one request to regenerate the missing value.

Without a lock:

Request A → Cache Miss → DB
Request B → Cache Miss → DB
Request C → Cache Miss → DB
Request D → Cache Miss → DB

With a lock:

Request A → Cache Miss → Acquire Lock → DB
Request B → Cache Miss → Wait
Request C → Cache Miss → Wait
Request D → Cache Miss → Wait

Request A → Store result in cache

B/C/D → Read newly populated cache

This is sometimes implemented using a distributed lock in Redis.

The important idea is:

Only one request should perform the expensive regeneration while other requests wait or use another fallback strategy.

This is particularly useful for expensive operations and highly popular keys.

2. Refresh the Cache Before It Expires

Instead of waiting for a popular cache key to expire, refresh it proactively.

For example, suppose:

TTL = 60 minutes

Rather than waiting until the 60th minute:

60 min → cache expires → requests hit DB

you can refresh the value before expiration:

55 min
   ↓
Background refresh
   ↓
New value stored
   ↓
TTL reset

This is often called cache warming or proactive cache refresh.

It works particularly well for data that:

• is frequently accessed
• is expensive to generate
• changes relatively infrequently

Examples include:

• application configuration
• feature flags
• popular products
• frequently accessed reports
• expensive dashboard data

3. Use Stale-While-Revalidate

Sometimes the best solution isn’t to force users to wait for fresh data.

Instead, you can temporarily serve a slightly stale value while refreshing the cache in the background.

Request
   ↓
Cached value exists but is slightly stale
   ↓
Return cached value immediately
   ↓
Background refresh
   ↓
Fresh value stored in cache

This approach can significantly reduce latency and prevent a large group of requests from simultaneously hitting the database.

The trade-off is that users may temporarily receive slightly stale data.

Therefore, this strategy is more suitable for data where a small amount of staleness is acceptable.

4. Update the Cache Along With the Write

Another strategy is to update the cache when the underlying data changes instead of simply deleting the cache entry.

For example:

UPDATE database
       ↓
Update cache
       ↓
Future reads → Cache

Compare that with cache invalidation:

UPDATE database
       ↓
Delete cache
       ↓
Next 100 requests
       ↓
Cache misses
       ↓
Database

Updating the cache can prevent this sudden wave of cache misses.

However, this introduces a consistency consideration.

What happens if:

Database update succeeds
        ↓
Cache update fails

Now the cache may contain stale data.

Therefore, this approach should be designed with appropriate retry, consistency, and failure-handling mechanisms.

How to Prevent Cache Avalanche

1. Add Randomness to TTL

One of the simplest ways to reduce synchronized expiration is to add TTL jitter.

Instead of:

Key A → 3600 sec
Key B → 3600 sec
Key C → 3600 sec
Key D → 3600 sec

use something like:

Key A → 3540 sec
Key B → 3680 sec
Key C → 3615 sec
Key D → 3720 sec

The expiration times are now spread out.

In code, the idea could be as simple as:

ttl = 3600 + random.randint(0, 600)

Now instead of thousands of keys expiring at exactly the same moment, their expiration is distributed over a time window.

This doesn’t eliminate cache misses.

It reduces the probability of synchronized cache misses.

2. Avoid Synchronized Cache Warming

Cache warming itself can cause problems if thousands of keys are populated simultaneously.

For example:

Deploy
  ↓
Warm 100,000 keys
  ↓
100,000 DB queries
  ↓
Database spike

If you need to warm a large cache, consider:

• batching
• rate limiting
• staggering requests
• prioritizing hot keys

The goal is to avoid replacing a cache problem with a database spike during cache warming.

3. Protect the Database

Caching shouldn’t be your only line of defense.

Even with a well-designed cache, you should still protect the database using mechanisms such as:

• connection pooling
• query timeouts
• rate limiting
• request throttling
• circuit breakers
• load shedding

This is important because a cache can fail.

Your system should still degrade gracefully when it does.

A Simple Production Example

Imagine an application with:

1000 requests/second

A popular endpoint normally gets its response from Redis.

1000 requests
      ↓
Redis
      ↓
~995 cache hits
      ↓
Only a few DB requests

Now the popular key expires.

Without protection:

1000 requests
      ↓
1000 cache misses
      ↓
1000 DB queries
      ↓
Connection pool saturation
      ↓
High DB latency
      ↓
API latency increases

With a regeneration lock:

1000 requests
      ↓
Cache miss
      ↓
1 request acquires lock
      ↓
1 DB query
      ↓
Cache populated
      ↓
Remaining requests read cache

The difference is enormous.

The cache didn’t make the system fragile.

The lack of a strategy for cache misses did.

A Practical Strategy

For a production system, you don’t necessarily have to choose only one technique.

A robust design might combine several:

Request
                       ↓
                    Cache
                       ↓
                ┌──────┴──────┐
                │             │
             Hit             Miss
                │             │
             Return      Check Lock
                              ↓
                    ┌─────────┴─────────┐
                    │                   │
                 Lock free           Lock exists
                    │                   │
                 Acquire              Wait
                    │                   │
                   DB                   │
                    │                   │
              Update Cache              │
                    │                   │
                    └─────────┬─────────┘
                              ↓
                         Return Data

For hot keys, you can combine this with:

• proactive refresh
• stale-while-revalidate
• TTL jitter
• database protection

What Should You Monitor?

Preventing cache stampede isn’t only about writing the right code.

You also need to know when it is happening.

Useful metrics include:

Cache metrics

• cache hit rate
• cache miss rate
• eviction rate
• key expiration rate
• cache latency

Application metrics

• request latency
• error rate
• requests per second
• concurrent requests

Database metrics

• CPU utilization
• active connections
• connection pool saturation
• query latency
• slow queries

A sudden combination like:

Cache hit rate ↓
Cache misses ↑
DB connections ↑
DB latency ↑
API latency ↑

should immediately make you think:

Could this be a cache-related load spike?

Final Takeaway

But a cache doesn’t remove the underlying work.

It simply changes when and where that work happens.

When the cache is available:

Request → Cache → Response

When the cache fails:

Request → Database → Expensive Work

At production scale, hundreds or thousands of requests can experience that failure at the same time.