🚀 Building a Real-World CI/CD Pipeline with Azure DevOps, Terraform & Ansible (EpicBook Project)

Modern software development is no longer just about writing code — it’s about automating deployment, infrastructure, and delivery pipelines so applications can be shipped faster and more reliably.

In this blog, I’ll walk through how I built a complete CI/CD pipeline using Azure DevOps, Terraform, and Ansible to deploy a full-stack application called EpicBook using a real-world DevOps architecture.

🧩 Project Overview

This project follows an enterprise-style two-repository model:

🔹 1. Infra Repository (Terraform)

Responsible for provisioning infrastructure:

• Resource Group
• Virtual Network & Subnets
• Virtual Machines (Frontend & Backend)
• MySQL Database
• Outputs (VM IP, DB host)

🔹 2. App Repository (Ansible + Application)

Responsible for application deployment:

• Configuring VMs
• Installing Nginx
• Deploying EpicBook application
• Connecting backend with MySQL

⚙️ CI/CD Architecture

The project uses two Azure DevOps pipelines:

🔹 Infra Pipeline

Automates infrastructure provisioning using Terraform:

• Install Terraform
• Authenticate using Azure Service Connection (SPN)
• Run:
• terraform init
• terraform plan
• terraform apply
• Generate outputs (VM IP, DB host)
• Publish outputs as artifacts

🔹 App Pipeline

Automates application deployment using Ansible:

• Install Ansible
• Fetch SSH keys from Secure Files
• Update inventory dynamically using Terraform outputs
• Deploy application on VMs
• Configure Nginx and backend connection

☁️ Azure DevOps Setup

To enable automation, I configured:

🔐 Service Principal (SPN)

Used for secure authentication between Azure DevOps and Azure.

🔗 Service Connection

Created in Azure DevOps to allow Terraform to provision resources securely.

🔑 Secure Files

SSH keys were stored securely in Azure DevOps:

• Used for VM login
• Avoided hardcoding credentials

📦 Variable Groups

Created epicbook-secrets to store sensitive data like:

• Database password (db_password)

🏗️ Infrastructure as Code (Terraform)

Terraform was used to automate infrastructure creation.

Key Features:

• Declarative infrastructure definition
• Azure VM provisioning
• Networking setup (VNet, Subnet)
• MySQL database setup
• Output variables for pipeline integration

Output Example:

• VM Public IP
• Database Host

These outputs were passed to the next pipeline stage for application deployment.

🤖 Configuration Management (Ansible)

Ansible handled configuration and deployment:

Tasks:

• Install Nginx on VMs
• Deploy EpicBook application files
• Configure backend database connection
• Ensure application is accessible via browser

🔄 End-to-End CI/CD Workflow

1. Code pushed to repository (GitHub/Azure Repo)
2. Infra pipeline triggers automatically
3. Terraform provisions Azure infrastructure
4. Outputs (IP, DB host) are generated
5. App pipeline consumes outputs
6. Ansible configures servers
7. Application is deployed successfully
8. EpicBook is accessible via public IP

⚠️ Challenges Faced & Fixes

During implementation, I faced several real-world DevOps issues:

❌ Agent Connectivity Issues

• Agent went offline or timed out
• Fixed by restarting self-hosted agent and ensuring stable WSL connection

❌ Service Connection Errors

• Fixed incorrect or mismatched service connection names

❌ Terraform Backend Issues

• Resolved storage account mismatch and invalid backend configuration

❌ Pipeline Blocking on sudo

• Removed sudo usage to avoid password prompts in automation

❌ SSH Key Configuration

• Corrected secure file setup in Azure DevOps

📚 Key Learnings

This project helped me understand real-world DevOps practices:

🔹 CI/CD Pipelines

• Automated infrastructure + application deployment
• Separation of concerns using multiple pipelines

🔹 Terraform (IaC)

• Infrastructure provisioning as code
• State management using Azure Storage backend

🔹 Ansible

• Configuration management and automation
• Remote server setup and deployment

🔹 Azure DevOps

• Pipeline creation and management
• Service connections & secure authentication
• Secure file and variable group usage

🔹 Troubleshooting Skills

• Debugging pipeline failures
• Fixing authentication and backend issues
• Handling real-time agent failures

🎯 Final Outcome

At the end of this project:

✔ Azure infrastructure was fully automated ✔ Application was deployed successfully ✔ CI/CD pipeline worked end-to-end ✔ No manual intervention required

This project helped me experience how real DevOps pipelines work in production environments.

🚀 Conclusion

Building this CI/CD pipeline gave me hands-on experience with:

• Cloud infrastructure automation
• Secure DevOps practices
• Real-world deployment workflows

It strengthened my understanding of how modern applications are built, deployed, and managed in scalable cloud environments.