Weighted Round-robin in ALB with dynamic weight in AWS

Introduction:

AWS comes with different built-in algorithms for load balancing algorithms. Although these algorithms are easy to use, there are times when we need to utilize custom algorithms for efficient traffic routing in cloud environments.

In this article, we will explore how we use a custom-built load balancing algorithm to route traffic to different EC2 instances to improve application performance and reliability.

The implementation of the weighted round-robin algorithm with dynamic weights (extension of weighted Round-robin algorithm existing in AWS)based on EC2 metrics obtained from CloudWatch will be done using boto3 (AWS SDK for Python).

Pre-requisites: Before proceeding, make sure you have the following:

• Configure OIDC
• Python 3.8 or higher and boto3 (AWS SDK for Python)

Implementation: The implementation code and the project folder structure can be found here: https://github.com/sandeep-tripathi/dynamic-alb-weighted-routing

How does AWS resources interact as per the algorithm:

1. OIDC for Authentication: OpenID Connect is an identity layer on top of OAuth 2.0 that allows clients to verify the identity of the end-user. In your scenario, OIDC is used to authenticate users or applications before they can interact with AWS services via a boto3 client.

• Authentication Process: In the code, OIDC authentication is used to obtain a token that allows an application to assume a specific AWS IAM role. This role then grants permissions to interact with AWS services, such as creating or modifying an Application Load Balancer (ALB).
• Boto3 Client Creation: Once authenticated, the token and assumed role credentials are used to create a boto3 client. This client is used to interact with AWS services like EC2, CloudWatch (for metrics), and the ALB.
• This implementation can be found in oidc_boto_authentication.py

2. Dynamic Weight Updates on Application Load Balancer (ALB):

This is the most crucial part, where we will create a combined utilization metric based on CPU and memory usage obtained from CloudWatch. Then after, we will use this to dynamically change the value of the weights, which will later be used for routing the traffic to a particular EC2 instance.

• Metrics Collection: The code uses CloudWatch to gather CPU and memory utilization metrics from the EC2 instances.
• Weight Calculation: These metrics are combined to calculate a combined utilization factor. The combination of the metrics is done simply by using their average.

Method in alb_weighted_routing.py for weight Calculation

Based on this factor, weights are assigned to each EC2 instance. Weight is then calculated by subtracting the combined_utilization with 100. We can also observe, weight is inversely proportional to combined_utilization.

This is the point in the code where you can play around with various mathematical equations to come up with the combined utilization of two metrics, CPU usage and Memory. Also, you can use any of the metrics that are available from CloudWatch instead of the two used here.

• Updating ALB: The code uses the boto3 client to update the target groups’ weights in the ALB. This ensures that the ALB routes traffic according to the latest performance metrics, with lower-utilized instances receiving more traffic.
• Auto Scaling Group (ASG): It performs the usual scaling up and down of the EC2 instance based on the EC2 instance metrics. It does not have any direct role in routing the traffic to a certain EC2 instance, but is a common best practice to have ASG.

3. Routing Traffic:

• ALB Decision-Making: The ALB’s behavior is managed programmatically through boto3. In the code, the weights assigned to different target groups (EC2 instances) are dynamically updated based on their CPU and memory utilization. This influences how traffic is routed to the EC2 instances behind the ALB.
• The traffic is routed to the EC2 instance with the lowest weight.

Architectural diagram: Weighted Round-robin algorithm with dynamic weights

• The ALB uses the updated weights to distribute incoming requests to the appropriate EC2 instances. The routing is influenced by the dynamic weights, ensuring that traffic is balanced in a way that optimizes resource utilization across the instances.

Output: During the execution of alb_weighted_routing.py, you should see iteration table printed in the console similar to the following:

Iteration 1:
+---------------------+------+---------------------+------------------------+---------------------------+---------+
| Instance ID         | Port | CPU Utilization (%) | Memory Utilization (%) | Combined Utilization Factor | Weight  |
+---------------------+------+---------------------+------------------------+---------------------------+---------+
| i-1234567890abcdef0 |  80  | 70.0                | 60.0                   | 65.0                       | 35.0    |
| i-abcdef1234567890  |  80  | 30.0                | 40.0                   | 35.0                       | 65.0    |
| i-0abcdef1234567890 |  80  | 50.0                | 80.0                   | 65.0                       | 35.0    |
+---------------------+------+---------------------+------------------------+---------------------------+---------+

Lowest Weight: 35.0 (Instance: i-1234567890abcdef0)   # Send traffic here...Weight = 100 - Combined Utilization Factor
Highest Weight: 65.0 (Instance: i-abcdef1234567890)

Routing Information
+---------------------+------+
| Instance ID         | Port |
+---------------------+------+
| i-1234567890abcdef0 |  80  |   # First traffic goes here
| i-abcdef1234567890  |  80  |
| i-0abcdef1234567890 |  80  |
+---------------------+------+

Target group weights updated and traffic routed successfully.

Here, it can be observed that the traffic is routed to the EC2 instance with the lowest weight. Just a note, the algorithm already uses the cyclic distribution of the traffic so despite two weights are same, tie-breaking is done through cyclic distribution (characteristic of Round-robin algorithm.)

Conclusion

In this article, we explored how we can use metrics collected in CloudWatch from EC2 instances to create custom metric and dynamically change the weights to route traffic to different EC2 instances based on the weight.

For this, OIDC can be used for securely authenticating and authorizing the application to interact with AWS resources, while the ALB dynamically routes traffic based on real-time performance metrics collected from EC2 instances from the CloudWatch.

References

• Load balancing algorithms in AWS
• Scaled and Load balanced application

About the Author: Sandeep Tripathi is a cloud architect with expertise in cloud computing and distributed systems. He enjoys exploring new technologies and sharing his knowledge through writing and speaking engagements. Connect with him on **LinkedIn**.