How to Prevent a Cache Stampede

Standard TTL-based caching works perfectly until you hit high-traffic keys.

The pattern is always the same: a worker checks the cache, finds a miss, queries the database, and updates the cache.

This model breaks down completely when a hot key expires under heavy load. The moment that key drops, hundreds or thousands of concurrent requests read the cache miss simultaneously. Every single one of those application threads then fires the exact same heavy query at your database at the same time.

This is a cache stampede, also called the thundering herd problem. It is not a slow degradation of performance. Your database CPU instantly spikes to 100%, connection pools exhaust, and the application layer begins throwing 504 gateway timeouts.

The Mental Model

To fix a cache stampede, you have to break the lockstep behavior of your application threads. The goal is simple: only one worker should compute the new value, while all other incoming requests either wait safely or read slightly stale data.

You are balancing consistency against availability. If your application can tolerate a few seconds of stale data, you can decouple the cache refresh from user requests entirely. If you need absolute consistency, you must force requests to line up behind a single worker.

The core architectural mistake is treating cache expiration as a hard drop off a cliff. Instead, you need a system that detects a key is about to expire, or is currently being updated, and handles concurrent traffic gracefully during that specific window.

Practical Mitigation Workflows

1. Mutex Locking (The Barrier Approach)

The most direct way to stop a stampede is to ensure only one thread can rebuild the cache at a time. When a cache miss occurs, the worker must acquire a lock before querying the database.

function getWithLock($key, $ttl) {
    $value = Cache::get($key);

    if ($value !== null) {
        return $value;
    }

    // Try to acquire a lock for 5 seconds
    $lock = Cache::lock($key . ':lock', 5);

    if ($lock->get()) {
        // Current thread won the race. Fetch and update.
        $value = fetchFromDatabase($key);
        Cache::put($key, $value, $ttl);
        $lock->release();

        return $value;
    }

    // Lost the race. Wait and retry reading from cache.
    sleep_ms(100);
    return getWithLock($key, $ttl);
}

Distributed locks introduce their own failure modes. If the worker holding the lock crashes or times out, you can stall your entire request pipeline. You must set strict, short timeouts on these locks to prevent cascading failures.

2. Probabilistic Early Expiration (XFetch)

A more elegant approach is to refresh the key before it actually expires, using a probabilistic algorithm. As the key nears its expiration time, the probability that a read request will trigger an early background refresh increases.

The standard formula for this is the XFetch algorithm:

Where delta is the delta time it takes to compute the value, and $\beta$ is an aggressive constant greater than 0.

function getWithXFetch($key, $beta = 1.0) {
    $item = Cache::getWithMetadata($key); // Returns ['value', 'ttl', 'delta']

    if (!$item) {
        return refreshAndCache($key);
    }

    // Calculate probabilistic early expiration
    if (-$beta * $item['delta'] * log(mt_rand() / mt_getrandmax()) > $item['ttl']) {
        // Trigger a background refresh or handle it inline
        return refreshAndCache($key);
    }

    return $item['value'];
}

This approach completely eliminates background stalls. The closer the key gets to expiration, or the longer the database query takes ($\delta$), the higher the chance a random request will step up and refresh the data early.

Trade-offs & Reality Checks

No mitigation strategy comes for free. Mutex locking adds latency for all the workers that lose the race, as they sit and poll the cache waiting for the winner to finish. If your database is slow, those waiting threads can balloon your web server’s process count and exhaust memory.

XFetch solves the latency problem but adds computational overhead. Running logarithmic math on every hot cache read uses CPU cycles. It also means you are actively refreshing keys that might not actually need to be kept alive, wasting database resources on low-value data if your traffic patterns shift.

Production Rules for Hot Keys

• Never use hard expirations for top-tier assets: If a query takes longer than 500ms and is hit hundreds of times per second, use a background cron job or a worker queue to refresh it on a strict schedule.
• Implement Stale-While-Revalidate: Configure your cache layer to return the expired value to the user while spinning up a background task to update the database. Old data is usually better than a broken page.
• Set aggressive lock timeouts: If you use distributed locks, never let a lock live longer than twice the maximum expected database query time.
• Add jitter to background crons: If you switch to cron-based generation, randomize the execution times slightly so your background workers do not hammer the database on the exact turn of the minute.