Understanding Load Balancers- Part 2

You are deploying a new e-commerce application with a web server tier (3 instances), an application server tier (3 instances), and a database tier (1 primary, 1 replica). All web and app servers need to be load-balanced.

Which load balancing algorithm would you initially choose for the web server tier, and why?

Initially, I would choose Round Robin for the web server tier.For a small number of instances (3 web servers), Round Robin is simple to configure and provides an even distribution of requests across all available servers.

As traffic grows or if specific performance bottlenecks emerge, I might consider switching to a more dynamic algorithm like Least Connections or Weighted Round Robin (if some servers have more capacity).

How would you configure health checks for the web servers to ensure only healthy instances receive traffic? What specific parameters would you monitor?

Health checks are crucial to ensure the load balancer only sends traffic to healthy instances.

Configuration:

• Protocol: HTTP (or HTTPS if SSL termination is at the load balancer or web server).
• Port: The port the web server is listening on (e.g., 80 for HTTP, 443 for HTTPS).
• Path: A specific, lightweight URL path that represents a healthy web server state (e.g., /healthz or /status). This path should ideally serve a static, small file or a simple "OK" response, without hitting the database or performing complex operations that could themselves fail.

Specific Parameters to Monitor:

• HTTP Status Code:
• Success Criteria: Expect an HTTP 200 OK status code. Any other 2xx status code might also be acceptable depending on the application's health check implementation, but 200 is standard.
• Failure Criteria: Any 3xx (redirect), 4xx (client error), or 5xx (server error) status code would indicate a problem.
• Response Time (Timeout): Define a maximum acceptable response time (e.g., 2–5 seconds). If the web server takes longer than this to respond to the health check, it’s considered unhealthy. This prevents sending traffic to a server that is slow or unresponsive.
• Interval: How often the load balancer performs health checks (e.g., every 5 seconds).
• Unhealthy Threshold (Consecutive Failures): The number of consecutive failed health checks before the server is marked as unhealthy and removed from the rotation (e.g., 2–3 failures). This prevents flapping (rapidly adding/removing servers due to transient issues).
• Healthy Threshold (Consecutive Successes): The number of consecutive successful health checks required before a previously unhealthy server is marked as healthy and re-added to the rotation (e.g., 2–3 successes).

If one of the web servers goes down, how quickly would the load balancer detect this and stop sending traffic to it? What happens when it comes back online?

Detection Time: The load balancer would detect a web server going down within the configured health check interval plus the unhealthy threshold. For example, if the interval is 5 seconds and the unhealthy threshold is 2, it would take approximately 10–15 seconds (2 consecutive failed checks + network latency) for the load balancer to detect the issue and stop sending traffic.

When it comes back online: The load balancer continues to perform health checks on the “down” server. Once the server becomes responsive and consistently passes the configured healthy threshold (e.g., 2–3 consecutive successful health checks), the load balancer will automatically mark it as healthy. Traffic will then be re-routed to the recovered web server according to the chosen load balancing algorithm (e.g., Round Robin will start sending new requests to it).

Which layer of the OSI model does a Load Balancer operate at to handle the database connections effectively, and which layer is required to handle the URL path-based routing? Briefly explain the fundamental difference in how they inspect traffic.

You have two types of traffic:

1. Database connections (TCP on a specific port).
2. Web traffic (HTTP/HTTPS) where you need to route requests based on the URL path (e.g., /api goes to API servers, and /images goes to a CDN or a separate storage cluster).

Database Connections: Requires a Layer 4 (L4) Load Balancer.

• L4 load balancing operates at the Transport Layer and inspects only the IP address and port in the network packets (TCP/UDP headers) to make routing decisions. It’s fast and efficient for simple connections like database traffic.

URL Path-Based Routing: Requires a Layer 7 (L7) Load Balancer.

• L7 load balancing operates at the Application Layer and can inspect the entire application content of the message, including HTTP headers, cookies, and the URL path. This deeper inspection allows for sophisticated routing rules based on the content of the request, like routing /api traffic to a dedicated API server farm.

You observe that users are losing their shopping cart contents when they navigate between pages. What might be the issue, and how would you address it using the load balancer?

Possible Issue:

The most likely issue is a lack of session persistence (or session affinity), also known as “sticky sessions,” combined with the application storing shopping cart contents directly in the web server’s local memory or file system.

When a user navigates between pages, their requests might be directed to different web servers by the Round Robin load balancer. If the shopping cart data is tied to the specific web server that initially received the request, and a subsequent request goes to a different web server, the new server won’t have that user’s session data, leading to a “lost” shopping cart.

How to Address it Using the Load Balancer:

While it’s generally recommended to handle session management at the application server tier or with a shared session store for scalability and resilience, you can address this at the load balancer level as a temporary or partial solution:

Enable Session Persistence (Sticky Sessions):

• Method: Configure the load balancer to use cookie-based session persistence.
• How it works: When a user’s first request arrives, the load balancer sends a cookie back to the client. This cookie contains information identifying the specific web server that handled that initial request. For all subsequent requests from that user, the load balancer reads this cookie and directs the traffic back to the same web server, ensuring session continuity.
• Load Balancer Configuration: You would typically find an option like “Session Persistence,” “Sticky Sessions,” or “Client Affinity” in your load balancer’s settings (e.g., AWS ELB/ALB, Nginx, HAProxy). You’d then select “Application Cookie” or “Load Balancer Generated Cookie.”

Important Considerations for Sticky Sessions:

• Load Distribution Imbalance: While it solves the cart issue, sticky sessions can lead to an uneven distribution of load if some users are highly active on one specific server.
• Server Failure Impact: If the “sticky” server goes down, the user’s session will still be lost, as their subsequent requests will be routed to a different server.
• Scalability Challenges: It makes horizontal scaling more complex, as sessions are tied to specific instances.

Recommended Long-Term Solution (Beyond Load Balancer):

The most robust and scalable solution for session management in an e-commerce application is to use a shared, centralized session store (e.g., Redis, Memcached, a dedicated database table for sessions). This allows any web server or application server to access the user’s session data, regardless of which instance initially handled the request. This eliminates the need for sticky sessions at the load balancer level and greatly improves resilience and scalability.