AWS EKS Blue/Green Deployment with Best Practices

About Client

The client has a global presence and provides worldwide aviation security services and assistance.

Challenge

The client’s prominent challenge was maintaining highly available infrastructure for critical APIs used across various airports.

In the realm of aviation, where downtime can translate into significant disruptions and inconveniences, the goal was to ensure seamless upgrades while maintaining operational excellence for airline check-in/checkout APIs.

Solution

Overview

In response to the critical need for zero downtime and a highly available platform, our team implemented an AWS Blue/Green Deployment strategy for the client. The project involved setting up two identical Amazon EKS clusters, each serving as a mirror of the other.

The Blue/Green strategy, coupled with Terraform Helm modules and IaC practices, played a pivotal role in achieving operational excellence and ensuring a reliable and highly available platform.

The implementation utilized the Amazon EKS Blueprint for Terraform, facilitating the creation of two clusters (eks-blue and eks-green) within the same VPC. These clusters leverage the AWS Load Balancer Controller and External-DNS add-ons for seamless application exposure. The Amazon EKS Blueprint automates cluster creation and add-on installation.

Client’s infrastructure blueprint

Outcomes

There is Zero downtime during upgrades and maintenance activities. Live traffic redirection to the updated environment occurred effortlessly, minimizing disruptions and ensuring continuous service availability.

The uniformity achieved across the Blue and Green clusters played a pivotal role in streamlining management and reducing the risk of discrepancies. This consistency was not only instrumental during routine operations but proved invaluable during upgrades and changes, ensuring a predictable and controlled deployment process.

The implementation of Route53 weighted policies for traffic management between the Blue and Green clusters proved highly effective. The strategy also provided a robust mechanism for testing and validating changes before exposing them to the entire user base.

The utilization of Terraform Helm modules for the seamless integration of critical EKS addons, combined with Infrastructure as Code (IaC) practices, contributed to a highly automated and efficient deployment process. This has not only reduced manual intervention but also provided a standardized and reproducible approach to managing the infrastructure. The incorporation of monitoring solutions and the Kuma server for proactive issue detection significantly bolstered the team’s ability to respond swiftly to potential anomalies. Alerts were triggered in real-time for issues such as downtime or high memory usage, enabling proactive intervention and resolution before these issues could impact the production environment.

Conclusion

The AWS Blue/Green Deployment strategy implemented within the Amazon EKS environment successfully achieved operational excellence and solidified the platform’s reliability. The platform now stands as a testament to the seamless integration of critical EKS addons, consistent and efficient deployment practices, proactive issue detection mechanisms, and, most importantly, the ability to perform upgrades and maintenance activities without disrupting services.

Where CloudDrove stood out

CloudDrove’s comprehensive solution stack encompasses a custom Terraform Helm module for AWS EKS addons, Infrastructure as Code with Terraform for environment management, Azure DevOps Pipelines for dynamic and automated deployments, CI/CD workflows for Terraform deployment, and rigorous static checks for code quality. This strategic combination of tools and practices ensures a robust, efficient, and reliable platform for critical check-in/checkout APIs.

Custom modules by CloudDrove

1. Terraform Helm Modules for Addons: CloudDrove has developed a custom Terraform EKS addons module explicitly tailored for AWS EKS addons, ensuring seamless integration of critical components such as Metrics Server, Monitoring Tools, AWS ALB Controller, EBS/EFS CSI Drivers, and KEDA. This module streamlines the deployment process, enhancing efficiency and consistency across environments.
2. The EKS cluster utilized in this Blue/Green Deployment was created using CloudDrove’s Terraform EKS module.

If you have different needs and configurations, you can check these examples to better understand the repo usage.

For hassle-free Cloud Management with DevOps at the center of the process, contact us at hello@clouddrove.com

About the author: Nilesh Gadgi, I like automating, building scalable pipelines, and improving and optimizing systems. I am a Strong Advocate for open source, Cloud computing, DevOps, Innovation and Automation.