Azure Load Balancer: From a Single Server to a Highly Available Architecture

Introduction

Before Load Balancer

When we first deploy an application, the architecture is usually simple.

One virtual machine hosts the frontend, another hosts the backend, and users directly access the frontend using its public IP address.

This setup works perfectly when traffic is low.

But as the application grows and thousands of users begin accessing it simultaneously, the architecture starts exposing several weaknesses.

Questions like these become common:

• What happens if the frontend VM crashes?
• How can we support ten times more users?
• How do we upgrade servers without downtime?
• How do large companies like Microsoft, Netflix, Amazon, and Google handle millions of requests every second?

The answer starts with understanding scaling and ultimately leads us to one of the most important networking services in Azure — Azure Load Balancer.

In this article, we’ll build that understanding step by step.

Initial Architecture (Before Load Balancer)

Imagine a very common architecture.

Users
   │
Public IP
   │
Frontend VM
   │
Backend VM

The frontend VM receives every incoming request. The backend VM handles databases, APIs, or business logic. At first glance, everything appears perfectly fine. Unfortunately, this architecture has several serious problems.

Problems We Faced

1. Single Point of Failure

Every request depends on a single frontend VM. If that VM crashes…

• Hardware failure
• OS crash
• Azure host maintenance
• Patch update

the entire application becomes unavailable. There is no backup server.

2. Limited Scalability

Suppose your VM has

• 2 vCPU
• 2 GB RAM

Initially, this may support a few hundred users. But as traffic increases,

• CPU reaches 100%
• Memory becomes exhausted
• Network bandwidth becomes saturated

Eventually users experience

• Slow pages
• Timeout errors
• Failed requests

3. No Traffic Distribution

Every request goes to exactly the same VM.

User 1
User 2
User 3
User 4
User 5
        │
        ▼
 Single Frontend VM

One machine handles everything.

4. Maintenance Downtime

Imagine upgrading Ubuntu. Or rebooting after Windows Updates. Since only one frontend VM exists, maintenance means downtime. Every user is affected.

5. Poor High Availability

High Availability means your application should remain accessible even if something fails. With a single server, availability depends entirely on one machine. That isn’t acceptable for production workloads.

First Solution That Comes to Mind — Scaling

Most engineers think,

“Let’s make the server more powerful.”

This approach is called Vertical Scaling.

Understanding Scaling

Scaling means increasing application capacity so it can handle more users and more traffic.

Azure supports two major scaling approaches.

• Vertical Scaling (Scale Up)
• Horizontal Scaling (Scale Out)

Let’s understand both.

Vertical Scaling (Scale Up)

Instead of adding more servers, we increase the resources of the existing server.

Vertical Scaling

Example

2 CPU
2 GB RAM

↓

4 CPU
8 GB RAM

↓

8 CPU
16 GB RAM

The application still runs on one VM.

Only its hardware becomes larger.

Advantages

• Very easy to implement
• No application architecture changes
• Good for small applications
• Quick performance improvement

Limitations

Every VM has a maximum hardware limit. Eventually you cannot add

• more CPU
• more RAM
• more storage

Also,

if the VM fails, the application still goes offline. Vertical Scaling does not eliminate the single point of failure.

Horizontal Scaling (Scale Out)

Horizontal Scaling

Instead of upgrading one VM, we create multiple VMs. Example

VM1

VM2

VM3

VM4

Now the application runs on multiple servers. This immediately improves

• availability
• scalability
• fault tolerance

However… another question appears.

A New Problem Appears

Now we have four frontend servers. How will users know which server to access?

New Problem Appears

User

?

VM1

VM2

VM3

VM4

Should users manually choose? Obviously not. We need a service that automatically distributes traffic. This is exactly why Load Balancers exist.

What is a Load Balancer?

A Load Balancer sits between users and backend servers. Instead of users connecting directly to servers, they connect to the Load Balancer. The Load Balancer intelligently distributes incoming requests across multiple healthy servers.

Load Balancer

Users

↓

Load Balancer

↓

VM1
VM2
VM3
VM4

No single server becomes overloaded.

Benefits of Using a Load Balancer

• High Availability
• Fault Tolerance
• Better Performance
• Horizontal Scaling
• Zero Downtime Deployments
• Improved Resource Utilization
• Automatic Traffic Distribution
• Better User Experience

Types of Azure Traffic Distribution Services

Azure provides multiple services for different networking scenarios. Choosing the correct service depends on your application’s requirements.

Deep Dive into Azure Load Balancer

Azure Load Balancer is a fully managed Layer 4 load balancing service.

It distributes TCP and UDP traffic across healthy backend resources within a region.

Instead of inspecting HTTP requests, it forwards packets based on IP addresses and ports, making it extremely fast and highly scalable.

At its core, Azure Load Balancer consists of four major components.

1. Frontend IP Configuration

The Frontend IP is the entry point for incoming traffic. Every client first connects here. Azure supports two types.

Public Frontend IP

Used when traffic comes from the Internet.

Example

Internet

↓

Public IP

↓

Load Balancer

Ideal for

• Public websites
• APIs
• Internet-facing applications

Private Frontend IP

Used only inside a Virtual Network or connected on-premises network.

Application Server

↓

Private IP

↓

Internal Load Balancer

Ideal for

• Internal APIs
• Backend services
• Microservices
• Enterprise applications

No Public IP is required.

2. Backend Pool

The Backend Pool contains the resources that actually serve requests.

It may include

• Azure Virtual Machines
• Virtual Machine Scale Sets (VMSS)
• IP-based backend targets (supported scenarios)

The Load Balancer distributes traffic only to resources in this pool.

3. Load Balancing Rules

Rules define how traffic should flow from the frontend to the backend.

A rule specifies:

• Frontend IP
• Frontend Port
• Backend Pool
• Backend Port
• Protocol (TCP/UDP)
• Health Probe
• Session Persistence (optional)
• Idle Timeout
• Floating IP (DSR) when required

Example:

• Client connects to Public IP on TCP port 80.
• The Load Balancer forwards traffic to the Backend Pool on port 80 using the configured rule.

Without a load-balancing rule, the Load Balancer does not know where to send traffic.

4. Health Probes

Health Probes continuously monitor the health of backend instances.

They can use:

• TCP
• HTTP
• HTTPS

At a configurable interval, Azure checks whether each backend instance is responding.

If an instance fails the configured threshold, Azure marks it as unhealthy and immediately removes it from rotation. New connections are sent only to healthy instances. Once the instance becomes healthy again, it is automatically added back to the Backend Pool.

This mechanism enables fault tolerance without manual intervention.

End-to-End Traffic Flow

Let’s see how everything works together.

User
        │
        ▼
Frontend IP
        │
        ▼
Azure Load Balancer
        │
        ▼
Load Balancing Rule
        │
        ▼
Health Probe Verification
        │
        ▼
Healthy Backend VM

If a backend VM becomes unhealthy, the Health Probe detects it, and the Load Balancer routes new traffic to the remaining healthy instances.

Best Practices

• Prefer Standard Load Balancer for production workloads.
• Use Availability Zones or VM Scale Sets for backend redundancy.
• Configure Health Probes carefully with appropriate intervals and thresholds.
• Restrict backend access using Network Security Groups (NSGs).
• Use Internal Load Balancer for private services and Public Load Balancer only when Internet access is required.
• Monitor metrics and diagnostics using Azure Monitor and Log Analytics.
• Test failover scenarios regularly to ensure backend resiliency.

Final Thoughts

Every scalable cloud application follows a similar evolution:

1. Start with a single server.
2. Traffic grows and exposes architectural limitations.
3. Vertical Scaling provides temporary relief but eventually reaches hardware limits.
4. Horizontal Scaling introduces multiple servers for greater capacity and resilience.
5. A Load Balancer becomes essential to intelligently distribute traffic, eliminate single points of failure, and improve availability.

Azure Load Balancer is the foundation of highly available, scalable infrastructure on Azure. By understanding its architecture — Frontend IP Configuration, Backend Pool, Load Balancing Rules, and Health Probes — you gain the building blocks required to design resilient production-grade applications.