🚦 Load Shedding: The Last Line of Defense

🚦 Load Shedding: The Last Line of Defense

Why Saving the System Means Saying “No”

SEO keywords: load shedding, Spring Boot overload protection, microservices resilience, backpressure, capacity control

When traffic spikes, most systems do one thing:

They try harder.

More threads More retries More CPU More pods

And that’s exactly how they die.

Healthy systems don’t try harder under overload. They shed load.

🚨 The Fatal Instinct: Be Helpful Under Pressure

Your service is overloaded.

What does it do?

• Accept more requests
• Queue them
• Slow down
• Time out

No errors. No alarms.

Just a slow collapse.

This is not resilience. This is politeness at scale.

🧠 What Load Shedding Actually Means

Load shedding is:

Intentionally rejecting work to preserve system health.

Not failure. Not rudeness.

Self-defense.

💣 Why Scaling Is Not a Solution

Auto-scaling:

• Reacts late
• Costs money
• Amplifies bad traffic
• Spreads failure

If your system collapses before scaling triggers, you’re already down.

⚠️ Where Overload Really Happens

Not at:

• CPU 100%
• Memory OOM

But at:

• Thread pool saturation
• Connection pool exhaustion
• Queue buildup
• Event loop blocking

By the time CPU is high, it’s too late.

🔁 Load Shedding vs Backpressure

Backpressure is polite.

Load shedding is necessary.

🧪 Simple Load Shedding in Spring Boot (Semaphore)

@Component
public class LoadSheddingFilter implements Filter {
private final Semaphore semaphore = new Semaphore(100);
    @Override
    public void doFilter(
            ServletRequest request,
            ServletResponse response,
            FilterChain chain)
            throws IOException, ServletException {
        if (!semaphore.tryAcquire()) {
            HttpServletResponse resp = (HttpServletResponse) response;
            resp.setStatus(429);
            resp.getWriter().write("Too Many Requests");
            return;
        }
        try {
            chain.doFilter(request, response);
        } finally {
            semaphore.release();
        }
    }
}

This:

• Caps concurrency
• Prevents thread exhaustion
• Keeps latency predictable

⚡ Rate Limiting Is Load Shedding

@Bean
public Bucket4jConfiguration bucketConfig() {
    return Bucket4jConfiguration.builder()
        .addLimit(Bandwidth.simple(100, Duration.ofSeconds(1)))
        .build();
}

429 responses are not errors.

They are signals.

🧠 Load Shedding in Reactive Systems

@GetMapping("/data")
public Mono<Response> getData() {
    return Mono.just("ok")
        .limitRate(100)
        .onErrorReturn(
            RejectedExecutionException.class,
            fallbackResponse()
        );
}

Reactive systems must shed load early — or event loops die.

💥 Why Load Shedding Saves Healthy Traffic

Without shedding:

• All users get slow responses

With shedding:

• Some users get fast responses
• System stays alive
• Recovery is instant

Partial failure is better than total failure.

📉 Metrics That Tell You When to Shed

Watch:

• Active thread count
• Queue depth
• Request latency slope
• Timeout frequency

Load shedding should trigger before timeouts spike.

🧠 Production Rule of Thumb

If you’re timing out:

You’re already late.

Load shedding should happen earlier.

🚀 Final Takeaway (The Line That Spreads)

Resilient systems don’t collapse gracefully. They refuse gracefully.

Saying “no” to some traffic is how you say “yes” to the system.