Scaling MCP Servers with SSE in Production: Solving Stateful Session Routing with ALB Sticky Sessions

1. Why Scaling MCP over SSE Introduced an Unexpected Problem

Everything worked perfectly in development.

Our MCP architecture was simple:

• One MCP client instance
• One MCP server instance
• SSE (Server-Sent Events) as the transport layer

Because there was only a single server handling all requests, we never had to think about session affinity, routing, or state ownership. Every request naturally landed on the same backend instance.

The situation changed when we moved to production scale.

As adoption increased, we expanded our deployment to multiple replicas:

• 3 MCP client instances
• 3 MCP server instances

Almost immediately, we started seeing intermittent failures. Conversations that had been established successfully would suddenly lose context. Requests that should have been associated with an existing session were being rejected by the server.

At first glance, nothing appeared wrong. The load balancer was functioning correctly, all server pods were healthy, and traffic was being distributed evenly across the cluster.

The root cause turned out to be a fundamental characteristic of running MCP over Server-Sent Events (SSE): SSE connections are inherently stateful.

When a conversation is established, the server instance that owns the SSE connection also owns the session state associated with that conversation. In a horizontally scaled environment, subsequent requests can be routed to a different server instance that has no knowledge of the original connection.

This article walks through the architecture decisions we made, why we intentionally chose SSE over Streamable HTTP, the scaling challenges we encountered, and the production solution we implemented using AWS Application Load Balancer (ALB) sticky sessions. I’ll also discuss the trade-offs of this approach and why it introduces limitations when integrating with third-party MCP clients such as Cursor and Claude.

If you’re planning to run MCP servers at scale, understanding these trade-offs early can save significant debugging and architectural rework later.

2. Quick MCP Transport Overview

Today MCP commonly supports:

SSE (Server Sent Events)

Client opens a long-lived connection.

Client -----------------------> Server
        Long-lived SSE Channel

The server maintains state associated with that connection.

Streamable HTTP

Each request is independent.

Request 1 -> Server
Request 2 -> Server
Request 3 -> Server

No persistent connection is required.

SSE

+--------+       Persistent Connection       +--------+
| Client | ================================> | Server |
+--------+                                   +--------+

Streamable HTTP

+--------+      Req1      +--------+
| Client | -------------> | Server |
+--------+                +--------+

+--------+      Req2      +--------+
| Client | -------------> | Server |
+--------+                +--------+

3. Why SSE Was the Right Choice for Our Use Case

For our use case, SSE provided several advantages.

1. Persistent Context

The MCP server could maintain state across the conversation lifecycle.

Instead of recreating context for every request, the existing session remained available.

2. Lower Session Management Complexity

With a persistent connection:

• Session initialization happens once
• Conversation state stays attached to the connection
• Fewer moving pieces

3. Real-Time Server Push

The server can push events immediately.

This is especially useful for:

• Tool execution updates
• Long-running workflows
• Progress notifications

4. Reduced Repeated Handshakes

With Streamable HTTP:

Request
Authenticate
Load Context
Process
Return

Repeat...

With SSE:

Connect Once
Reuse Session
Exchange Events

4. What We Would Lose with Streamable HTTP

Why We Didn’t Simply Switch to Streamable HTTP

When we evaluated Streamable HTTP, the main concern wasn’t functionality. It was operational complexity.

Loss of Connection Affinity

Every request becomes independent. The server must reconstruct context repeatedly.

Additional State Management

Instead of relying on a connection:

Connection
    ↓
Conversation State

you need:

External Session Store
    ↓
Redis / Database
    ↓
Lookup Per Request

SSE

Client
   |
Persistent Connection
   |
Server (holds session state)

Streamable HTTP

Client
   |
HTTP Request
   |
Load Balancer
   |
Server
   |
Redis Session Store

Increased Latency

Every request may require:

• Session lookup
• Context retrieval
• State hydration

before actual processing begins.

More Infrastructure

Typical Streamable HTTP deployments often require:

• Redis
• Session synchronization
• Distributed state management

SSE allowed us to avoid much of that complexity.

5. The Problem That Emerged After Horizontal Scaling

Everything Broke When We Scaled

Initially:

1 Client
1 Server

There was nowhere else for traffic to go. Every request naturally landed on the same server.

After scaling:

3 Client Pods
3 Server Pods

Traffic started flowing through the load balancer.

                 +------------+
                 |    ALB     |
                 +------------+
                 /     |      \
                /      |       \
               /       |        \
          Server1  Server2   Server3

The issue was subtle.

The SSE connection was established with:

Server-1

However later requests could be routed to:

Server-2

OR

Server-3

Those servers had no knowledge of the original SSE session.

Result:

Session Not Found
Connection Missing
Conversation Context Lost

6. Understanding Why Requests Started Failing

Understanding the Root Cause

SSE sessions are effectively tied to a specific backend instance.

Example:

Conversation A
    |
    +--> Connected to Server-1

Server-1 owns:

• session state
• active connection
• conversation lifecycle

If a subsequent request lands on Server-2:

Conversation A
    |
    +--> Request routed to Server-2

Server-2 cannot access the live SSE connection maintained by Server-1.

The request fails.

Step 1

Client
   |
   | SSE Connect
   v
Server-1

Step 2

Client
   |
   | Follow-up Request
   v
ALB
   |
   +----> Server-2

Server-2:
"I don't know this session"

7. Solving Session Affinity with AWS ALB Sticky Sessions

Solving It with AWS ALB Sticky Sessions

To preserve session affinity, we enabled sticky sessions on the AWS Application Load Balancer.

When the SSE connection is first established:

Client --> ALB --> Server-1

The ALB generates a cookie.

Example:

AWSALB=xyz123

The client stores this cookie.

For every request belonging to the same conversation:

Cookie: AWSALB=xyz123

is sent back to the ALB.

The ALB then guarantees routing to the same backend instance.

Conversation A

Request 1 --> Server-1
Request 2 --> Server-1
Request 3 --> Server-1
Request 4 --> Server-1

                 +----------------+
                 |      ALB       |
                 +----------------+
                        |
                 AWSALB Cookie
                        |
                        v
                    Server-1
                        ^
                        |
                All Requests
                Same Conversation

FLOW

1. SSE connection created

Client
  |
  +--> ALB
          |
          +--> Server-1

2. ALB issues cookie

AWSALB=session123

3. Client stores cookie

4. Every subsequent request

Cookie: AWSALB=session123

5. ALB routes request back to Server-1

6. Session remains valid

8. Client Changes Required for Sticky Session Routing

The load balancer alone is not enough.

The client must ensure that:

1. The ALB cookie is captured.
2. The cookie is stored against the conversation/session.
3. Every request belonging to that conversation sends the same cookie.

Pseudo-flow:

Conversation Start
       |
Capture ALB Cookie
       |
Store Cookie
       |
Conversation Continues
       |
Attach Cookie
       |
ALB Routes To Same Server

Without this step, sticky sessions will not work reliably.

9. Where This Approach Falls Short

Why This Doesn’t Work for Cursor or Claude Integrations

Our solution assumes we control the MCP client implementation.

Because we own the client, we can:

• Capture ALB cookies
• Persist them
• Send them back

Tools like:

• Cursor
• Claude Desktop

operate as external MCP clients.

We cannot guarantee that they:

• Preserve ALB cookies
• Expose cookie management APIs
• Send sticky-session cookies back on subsequent requests

As a result:

Cursor
   |
   +--> ALB
            |
      Request 1 -> Server-1

      Request 2 -> Server-3

      Request 3 -> Server-2

The session affinity guarantee disappears.

    Cursor
      |
      v
     ALB
   / | \
  /  |  \
 /   |   \
S1   S2  S3

No guarantee that all
requests hit same server

10. Alternative Architectures We Considered

Long-Term Alternatives

While sticky sessions solved our production problem, there are more scalable approaches.

Option 1: Shared Session Store

Store session state in Redis.

Server-1
Server-2
Server-3
      |
      v
    Redis

Any server can process any request.

Option 2: Streamable HTTP

Move away from connection-bound state entirely.

Trade-offs:

Pros:

• Easier horizontal scaling
• No session affinity
• Better compatibility with external MCP clients

Cons:

• Externalized state management
• Additional infrastructure
• Increased request complexity

Option 3: Distributed Session Layer

Build a dedicated session service.

Useful for very large MCP deployments.

Key Takeaways from Running SSE-Based MCP at Scale

SSE gave us a simple and efficient way to maintain conversational state in our MCP deployment. The challenge only appeared when we horizontally scaled the platform. Because SSE sessions are tied to specific backend instances, requests started reaching servers that did not own the original connection. We solved this using AWS ALB sticky sessions and cookie propagation, ensuring all requests for a conversation consistently reached the same server. While this approach worked well for our controlled clients, it highlights an important limitation: integrations such as Cursor or Claude cannot always participate in load-balancer affinity strategies. Teams adopting SSE for MCP should plan their scaling strategy early and carefully evaluate whether connection affinity, shared session storage, or Streamable HTTP is the best fit for their ecosystem.

Additional Section: Decision Matrix

SSE + Sticky Sessions vs Streamable HTTP + Shared State

When evaluating MCP deployment architectures, we considered two primary approaches:

Option 1: SSE + Sticky Sessions

Client
   |
   v
ALB (Sticky Session)
   |
   +------> Server-1
              |
          Session State

Option 2: Streamable HTTP + Redis

Client
   |
   v
Load Balancer
   |
   +--> Server-1
   |
   +--> Server-2
   |
   +--> Server-3
            |
            v
          Redis

| Category                      | SSE + Sticky Sessions | Streamable HTTP + Redis  |
| ----------------------------- | --------------------- | ------------------------ |
| Initial Complexity            | Low                   | Medium                   |
| Infrastructure Components     | ALB + MCP Servers     | LB + MCP Servers + Redis |
| Horizontal Scaling            | Moderate              | Excellent                |
| Session Affinity Required     | Yes                   | No                       |
| Shared State Store Required   | No                    | Yes                      |
| Request Routing Flexibility   | Low                   | High                     |
| Operational Overhead          | Low                   | Medium                   |
| Latency                       | Lower                 | Slightly Higher          |
| External Client Compatibility | Limited               | Excellent                |
| Cursor Compatibility          | Potential Issues      | Works Naturally          |
| Claude Compatibility          | Potential Issues      | Works Naturally          |
| Failure Recovery              | Harder                | Easier                   |
| Pod Replacement Handling      | More Complex          | Simpler                  |