How Spring Boot Actually Handles 100,000 RPS (Deep Dive Into WebFlux + Netty)

Every backend engineer hits this point.

The dashboard turns red. Your API latency spikes. Your HPA keeps doubling pods. CPU hits 90%. Requests begin queuing. Throttling starts. Team members yell, “Scale up!! Increase replicas!! Increase nodes!!”

Until someone asks the real question:

“Why is our backend dying at just 8,000 RPS?”

And then you discover a truth that breaks everything you believed:

Your backend isn’t slow because it needs more pods. It’s slow because it’s BLOCKING.

This is the moment every engineer realizes:

• Tomcat can’t save you.
• More pods can’t save you.
• Vertical autoscaling won’t save you.

But one thing can:

A non-blocking, event-loop based design — WebFlux running on Netty.

Because with the right architecture,

Spring Boot can push past 100,000 RPS on a single 8-core machine.

Let’s break down how.

This is the deepest, most low-level explanation of WebFlux + Netty you’ll find — without the useless theory.

Why Tomcat (Spring MVC) Breaks Before 10,000 RPS

Tomcat uses a model called:

Thread-Per-Request (TPR)

Meaning:

1 Request ⇒ 1 Thread

Now imagine 10,000 concurrent requests.

Tomcat tries to create (or maintain):

~200 - 2000 threads

And this creates four massive problems:

1. Context Switching Explosion

Threads constantly switch in/out of CPU cores.

• Switching cost ↑
• Latency ↑
• Throughput ↓

2. Thread Starvation

When all thread are busy:

• Requests wait
• Queues form
• Timeouts begin
• Clients retry
• API collapses

3. Blocking IO Halts the System

If your code does something like:

• JDBC query
• External API call
• File IO
• Redis call
• Thread.sleep
• Heavy logging

The thread stays blocked, wasting CPU cycles.

This is why Tomcat cannot scale beyond a limit.

4.Memory Overhead

Each thread reserves:

• Stack Memory
• ThreadLocal Memory
• Task queue

More thread → more RAM → GC pressure → full GC pauses → meltdown.

So How Does WebFlux + Netty Handles 100,000 RPS?

The WebFlux + Netty Execution Model: Event Loops

Netty — the engine under Spring WebFlux — uses event loops, not thread per request.

Think of them like this:

8 CPU cores → 8 event loop threads

That’s it. No 2000 threads. No chaos.

Here’s what happens:

A single event loop thread handles:

• Accepting connections
• Parsing requests
• Running callbacks
• Writing responses
• Scheduling small tasks

While staying non-blocking.

The Magic: Non-Blocking IO (NIO)

Instead of waiting for IO to complete:

Netty asks the OS to notify it when data is ready.

This avoids blocking threads.

Diagram:

Because nothing blocks, one thread can handle tens of thousands of concurrent requests.

Tomcat vs Netty (REAL difference)

Tomcat (blocking)

10,000 requests = 10,000 waiting threads = meltdown

Netty (non-blocking)

10,000 requests = 8 threads = completely stable

How WebFlux Works Internally

Important components:

1. Netty ChannelPipeline

When a requests arrives:

Socket → Channel → Pipeline → Handlers → WebFlux Dispatcher

Each steps is asynchronous.

2. Reactor (Mono/Flux) Pipeline

Inside your controller:

@GetMapping("/fast")
public Mono<String> fast() {
    return Mono.just("ok");
}

This creates a pipeline:

Publisher → Operator → Subscriber

Each callback runs on event loops, not random threads.

3. Backpressure

If your server os overloaded:

• WebFlux slows down data consumption
• Prevents overload
• Maintains stable latency

Sequence Diagram: How One Request Flows

Meanwhile, the same event loop handles thousands of requests in between.

Why This Handles 100,000 RPS

Because:

• No threads wait
• No context switching
• No blocking
• Minimal memory usage
• CPU stays busy doing actual work
• Backpressure avoids overload

But Here’s the TRUTH: Your WebFlux App Will NOT Hit 100K RPS If…

WebFlux is not magic.

It collapses instantly if you put any blocking code on the event loop.

Here are the biggest killers:

Killer #1: JDBC (Blocking DB Drivers)

JDBC is blocking. PostgreSQL is blocking. MySQL is blocking.

If you query DB directly from WebFlux, the event loop stops, killing throughput.

Solution:

• Move to R2DBC (non-blocking database driver)

OR

• Offload to boundedElastic:

Mono.fromCallable(() -> jdbcRepo.findById(id))
    .subscribeOn(Schedulers.boundedElastic());

Killer #2: External REST Calls

Most HTTP clients (Apache, OkHttp) are blocking.

Use:

WebClient (reactive), not RestTemplate.

Killer #3: Heavy CPU computation on Event Loop

Example:

int result = heavyComputation();  // ❌

This blocks the event loop.

Fix:

Mono.fromCallable(() -> heavyComputation())
    .subscribeOn(Schedulers.parallel());

Killer #4: Logging per request

If you log:

• request body
• response
• trace id
• headers
• every call

You will destroy your event loops.

Benchmarks: What 100K RPS Actually Looks Like

Machine:

• 8 vCPU
• 16 GB RAM
• Java 17
• Spring WebFlux
• Netty
• No blocking code

Load test: wrk -t12 -c20000 -d30s

Results:

Metric. Value

RPS. 103,000/sec

p50. 1.8 ms

p95. 4.2 ms

Threads. ~14 total

CPU. ~85%

Memory usage. 800MB

When You SHOULD NOT Use WebFlux

If your app has:

• Heavy relational database logic
• Multiple blocking APIs
• Lots of file IO
• CPU-heavy workloads
• Complex synchronous flows

Then WebFlux will not hit 100K RPS.

In fact —

Tomcat might perform better for those use cases.

WebFlux is ideal for:

• Proxy servers
• Gateways
• Real-time push APIs
• Streaming
• High-throughput stateless APIs
• Lightweight ML inference
• Long polling
• Notification fan-out

Final Takeaway:

100,000 RPS Is Less About Hardware…

More About Architecture**

Most engineers believe scaling is expensive.

But the truth is:

“You don’t need more pods.

You need fewer blockers.”

WebFlux + Netty + event loops make Spring Boot insanely fast — but only when you follow the rules of reactive programming.

Get the threading model right,

avoid blocking,

enable backpressure,

and Spring can outperform Node.js, Go, and even Rust in certain workloads.

100,000 requests/second is not a dream — it’s an architecture decision.

Netty (non-blocking)