System Design — #1 Scalability

Scalability is the ability of a system to handle more work or more users without breaking down.

Let us say we have like 1000 users for an application, and after some days, many people started using it, and now it has millions of users. For the system to handle these millions of users, we need it to be scalable.

In this article, we will learn about some of the techniques to make an application scalable. We will go through Horizontal scaling, Vertical Scaling, Stateful and stateless systems, which scaling do they prefer.

1. Horizontal Scaling: Horizontal scaling (aka scaling out) refers to adding additional nodes or machines to your system/infrastructure to cope with new demands.

• Example: Add or reduce the number of virtual machines (VMs) in a cluster of VMs
• It’s often considered the most effective way to scale for large systems.
• Concurrency here is achieved by distributing multiple jobs across multiple machines over the network at a time. This reduces the workload on each machine.

Advantages of Horizontal scaling:

1. No single point of failure: even if one server fails, others will process the request. This can be addressed as increased fault tolerance (Fault tolerance is a system’s built-in ability to continue operating correctly and without interruption even when one or more of its components fail)
2. Increased capacity: More nodes or instances can handle a larger number of incoming requests, and there is a lower chance that a server will get overloaded with requests.
3. With methods like autoscaling, servers can be utilized properly without underutilization
4. Less Downtime: Since you’re adding servers, the old ones stay running. If done right, downtime is avoidable

Disadvantages of Horizontal scaling:

1. More maintenance and complexity: Managing multiple servers requires added tools for load balancing, virtualization, and backups. You also need to ensure that nodes sync and communicate properly.
2. Higher initial costs: Adding new servers is more expensive than upgrading existing ones.

2. Vertical scaling: Vertical scaling (aka scaling up) describes adding more power to your current machine.

• Example: Upgrading a MySQL server from 16 GB RAM to 64 GB to handle more queries.
• Concurrency here is achieved by multi-threading on the existing machine to handle multiple requests at the same time.

Advantages of Vertical Scaling:

1. Cost-effective: Cheaper than new hardware, less need for backup or virtualization tools, and easier to manage since it is a single server
2. Less complex process communication: One node handles everything; no syncing needed.
3. Less Maintenance: Since you only have one machine to manage, you don’t need complex tools for load balancing, syncing, or backups across many nodes.

Disadvantages of Vertical Scaling:

1. Hardware Limits: Every server has a maximum capacity (max CPU sockets, max RAM slots, max storage). After a point, you simply can’t upgrade further.

Example: If a server supports a maximum of 1 TB RAM, you can’t go beyond that, even if your workload grows.

Why horizontal scaling wins here: You can keep adding more servers indefinitely

2. Single Point of Failure: Since everything runs on one machine, if the server goes down, the entire system goes down. Recovery may take hours or days.

Why horizontal scaling wins: Even if one server fails, others keep working.

3. Downtime During Upgrade: To scale vertically, you usually need to shut down the server to add RAM, CPU, or replace storage. This causes downtime of the server, and the application will not be able to respond.

Why horizontal scaling wins: You can add/remove servers without stopping the whole system.

Before moving on to further discussion about horizontal scaling and vertical scaling, let us look into what stateful and stateless systems are.

Stateless systems: The server does not remember anything about the client between requests. (No session storage on the server) . Each request is treated as an independent transaction. Every request from the client must contain all the information needed for the server to process it.

Example: Rest API Calls- Each request carries authentication tokens and data; the server doesn’t remember you after sending the response

Compare it to this analogy: Ordering food at a fast-food counter. Every time you order, you say exactly what you want – the cashier doesn’t remember you.

Stateful systems: The server remembers information (state) about the client across multiple requests. The server stores session data (like login status, shopping cart, or ongoing transactions) and uses it when handling future requests

Here, Requests are dependent on previous interactions.

Example: Traditional banking systems (server remembers your session)

Relate it to this example: This system is like a restaurant waiter. Once you sit down, the waiter remembers your table, your drink order, and your food preferences until you leave; you don't need to mention your table number every time you order food.

Stateful systems often rely on vertical scaling. Here’s why:

In a stateful setup, the server keeps client context (session, transactions, game state, etc.) in its local memory. That means if the client’s next request goes to another server, that server won’t know the context unless you build session replication (complex and expensive). So, to keep things simple, we usually use vertical scaling

Does it mean that horizontal scaling is not possible for stateful systems? No, it is possible. But it requires extra coordination overhead. Vertical scaling is the natural choice because the state already lives inside the server.

How stateful systems can scale horizontally (with trade-offs):

1. Sticky Sessions (Session Affinity): Use a load balancer that ensures each client is always routed to the same server where its state lives. Example: A user starts a shopping cart session on Server A; all future requests from that user always hit Server A. Trade-offs: Uneven load distribution (some servers overloaded, others underutilized). If that server crashes, the user session is lost (unless backed up).
2. Replication Across Servers: Actively replicate session state between. servers (so any server can handle any request). Example: Multiplayer game servers replicating game state across regions. Trade-offs: High network overhead, Hard to maintain strong consistency (risk of race conditions).

Stateless systems prefer horizontal scaling. Here’s why:

In a stateless system, every request is independent and carries all the info needed. The server doesn’t have to remember sessions → no server-specific state. This means any server can handle any request without special coordination, also making load balancing simple, fault tolerance high, and scaling as easy as adding more servers. This decoupling of state from servers eliminates the need for sticky sessions or replication, which makes horizontal scaling the most efficient choice.

If you have reached here, Congrats, you have successfully learnt 2 most effective ways to make an application scalable. Stay tuned for more such articles. Cheers!

Resources:

[embed]Horizontal Vs. Vertical Scaling: Which Should You Choose? Compare horizontal vs. vertical scaling - both in the cloud and on-premise - and discover which one is best for your…www.cloudzero.com

[embed]Stateful vs. Stateless Architecture When a client interacts with a server, there are two ways to handle it:blog.algomaster.io