Moving from Static Inventory to Ansible Dynamic Inventory Using aws_ec2

Introduction

In traditional Ansible automation, hosts are defined statically in an inventory file.

However, in cloud environments like AWS, infrastructure is dynamic — EC2 instances are frequently created, terminated, or replaced. Maintaining a static inventory in such environments quickly becomes error-prone and unscalable.

In our case, every Auto Scaling replacement meant updating inventory files manually, which often caused failed Ansible runs.

This is where Ansible Dynamic Inventory comes into play.

In this article, we will understand:

• what dynamic inventory actually means in AWS
• how the aws_ec2 plugin builds inventory behind the scenes
• what needs to be installed and why
• Why each requirement is needed
• Finally, How Ansible dynamic inventory is created with a live demo

What Is Ansible Dynamic Inventory?

Dynamic inventory allows Ansible to fetch host information at runtime from an external source such as: AWS EC2/Azure/GCP or Kubernetes

Instead of hardcoding servers, Ansible queries AWS APIs and builds the inventory dynamically every time a playbook runs.

In short, Ansible stops asking us where the servers are, and AWS becomes the source of truth.

What Is aws_ec2 Plugin?

The aws_ec2 plugin is an official Ansible inventory plugin provided by the amazon.aws collection.

It:

• Connects to AWS using boto3
• Calls DescribeInstances API
• Collects EC2 metadata
• Builds inventory groups dynamically

In modern Ansible versions (2.10+), this plugin is shipped via the amazon.aws collection

How aws_ec2 Dynamic Inventory Works (Internals)

High-level flow:

1. Ansible reads aws_ec2.yml
2. Plugin loads AWS credentials and calls AWS EC2 APIs via boto3
3. Fetches EC2 instance metadata
4. Applies filters (region, tags, state)
5. Groups instances dynamically
6. Assigns host variables (IP, tags, instance type)
7. Inventory is ready for playbook execution

This happens every time we run Ansible.

Required Components

1. amazon.aws Collection

$ ansible-galaxy collection install amazon.aws

• It provides AWS modules and inventory plugins like aws_ec2

2. AWS credentials

• These credentials are required so Ansible can call AWS EC2 APIs securely.

3. Python Library — boto3

$ pip3 install boto3

• boto3 is used internally by the plugin to communicate with AWS

4. aws_ec2.yml inventory file

• It Defines how inventory should be built
• Controls grouping, filtering, and host variables

Let us try to understand ansible dynamic inventory creation with a live demo:

✅ Demo Overview

In this demo setup:

• Using Terraform, we create a production environment with EC2 instances (clients) using pre-created AMI.
• Terraform is used here to ensure the infrastructure itself is reproducible and not manually created.
• Verify ansible inventory using aws_ec2 plugin in a graph
• Run ansible-playbook to install nginx in ansible client servers using aws ansible tags & keyed_groups

🧪 Hands-On Practical Steps

Pre-requisite:

• A custom AMI was created that includes Ansible and other required tools like unzip, jq, net-tools etc.
• An Ansible controller EC2 instance is manually created and used to run Terraform and Ansible commands.

1. Clone the github project which contains terraform configuration files to provision: VPC, Subnets, VPC Peering, EC2 instances- https://github.com/SabirAliShaik/Ansible-Terraform-Testing.git
2. Provision the production environment with 3 EC2 instances
3. Install required dependencies — python boto3, amazon.aws collection using ansible-galaxy, aws_cli
4. Configure ansible inventory file — aws_ec2.yml in ansible.cfg
5. Execute ansible-inventory commands and Verify inventory using ansible-inventory — graph
6. Execute ansible-playbook to install nginx on ansible client servers using tags created from ansible inventory.

Implementation:

1. Clone the github project which contains terraform configuration files to provision: VPC, Subnets, VPC Peering, EC2 instances-

$ git clone https://github.com/SabirAliShaik/Ansible-Terraform-Testing.git

2. Provision the production environment with 3 EC2 instances

$ cd Ansible-Terraform-Testing

$ terraform init

$ terraform apply --var-file 15.terraform.tfvars

aws_vpc.default: Creating...
aws_vpc.default: Still creating... [10s elapsed]
aws_vpc.default: Creation complete after 12s [id=vpc-06c607a93bb607bc5]
aws_vpc_peering_connection.ansible-vpc-peering: Creating...
aws_security_group.allow_all: Creating...
aws_internet_gateway.default: Creating...
aws_subnet.public-subnets[0]: Creating...
aws_route_table.terraform-public: Creating...
aws_subnet.private-subnets[2]: Creating...
aws_route_table.terraform-private: Creating...
aws_subnet.private-subnets[1]: Creating...
aws_subnet.public-subnets[1]: Creating...
aws_subnet.public-subnets[2]: Creating...
aws_internet_gateway.default: Creation complete after 0s [id=igw-0b0b07fae676b992d]
aws_subnet.private-subnets[0]: Creating...
aws_route_table.terraform-public: Creation complete after 0s [id=rtb-0b413b2ad8ddbf2f1]
aws_route.igw-route: Creating...
aws_route_table.terraform-private: Creation complete after 0s [id=rtb-0ce251273cc365652]
aws_subnet.private-subnets[1]: Creation complete after 1s [id=subnet-0d409635a16fdd716]
aws_subnet.private-subnets[2]: Creation complete after 1s [id=subnet-048dade13cd35a889]
aws_route.igw-route: Creation complete after 1s [id=r-rtb-0b413b2ad8ddbf2f11080289494]
aws_subnet.private-subnets[0]: Creation complete after 1s [id=subnet-0af445f7c3471b9f7]
aws_route_table_association.terraform-private[2]: Creating...
aws_route_table_association.terraform-private[1]: Creating...
aws_route_table_association.terraform-private[0]: Creating...
aws_vpc_peering_connection.ansible-vpc-peering: Creation complete after 1s [id=pcx-07130731a5c18ed04]
aws_route.peering-from-ansible-vpc: Creating...
aws_route.peering-to-ansible-vpc: Creating...
aws_route_table_association.terraform-private[2]: Creation complete after 0s [id=rtbassoc-07b2692fa3d5be0ba]
aws_route_table_association.terraform-private[0]: Creation complete after 0s [id=rtbassoc-0ea2574e34121f97a]
aws_route_table_association.terraform-private[1]: Creation complete after 0s [id=rtbassoc-00d9452c4f3040f3f]
aws_route.peering-to-ansible-vpc: Creation complete after 1s [id=r-rtb-0b413b2ad8ddbf2f13854007479]
aws_route.peering-from-ansible-vpc: Creation complete after 1s [id=r-rtb-0578bf53fdd15cf2b986176854]
aws_security_group.allow_all: Creation complete after 2s [id=sg-0360deecfd3a6c3db]
aws_subnet.public-subnets[0]: Creation complete after 3s [id=subnet-01b22ff823f8b108a]
aws_subnet.public-subnets[1]: Creation complete after 4s [id=subnet-041613b156d5f675d]
aws_subnet.public-subnets[2]: Creation complete after 4s [id=subnet-0512aad3e232fdda6]
aws_route_table_association.terraform-public[0]: Creating...
aws_route_table_association.terraform-public[2]: Creating...
aws_route_table_association.terraform-public[1]: Creating...
aws_instance.webservers[0]: Creating...
aws_instance.webservers[2]: Creating...
aws_instance.webservers[1]: Creating...
aws_route_table_association.terraform-public[2]: Creation complete after 0s [id=rtbassoc-0f335b74503c9d9b2]
aws_route_table_association.terraform-public[1]: Creation complete after 0s [id=rtbassoc-08bb286be6e4a6b7a]
aws_route_table_association.terraform-public[0]: Creation complete after 0s [id=rtbassoc-0f16b7b1d73f594d5]
aws_instance.webservers[2]: Still creating... [10s elapsed]
aws_instance.webservers[0]: Still creating... [10s elapsed]
aws_instance.webservers[1]: Still creating... [10s elapsed]
aws_instance.webservers[1]: Creation complete after 12s [id=i-02dcdbf2813907006]
aws_instance.webservers[2]: Creation complete after 12s [id=i-0c09a153988935e5b]
aws_instance.webservers[0]: Still creating... [20s elapsed]
aws_instance.webservers[0]: Creation complete after 21s [id=i-094462ba230508761]
local_file.ansible-inventory-file-yaml: Creating...
local_file.ansible-inventory-file: Creating...
local_file.ansible-inventory-file-yaml: Creation complete after 0s [id=f105191b0375bde6b850b6dc671304bb44b3aca6]
local_file.ansible-inventory-file: Creation complete after 0s [id=8e3f19440c6a2d25283e5128460ff8ad78ad49e4]
null_resource.webservers: Creating...
null_resource.webservers: Provisioning with 'local-exec'...
null_resource.webservers (local-exec): Executing: ["/bin/sh" "-c" "      sleep 10\n      ansible -i invfile pvt -m ping\n"]
null_resource.webservers: Still creating... [10s elapsed]
null_resource.webservers (local-exec): [WARNING]: Host 'testserver03' is using the discovered Python interpreter at '/usr/bin/python3.12', but future installation of another Python interpreter could cause a different interpreter to be discovered. See https://docs.ansible.com/ansible-core/2.19/reference_appendices/interpreter_discovery.html for more information.
null_resource.webservers (local-exec): [WARNING]: Host 'testserver02' is using the discovered Python interpreter at '/usr/bin/python3.12', but future installation of another Python interpreter could cause a different interpreter to be discovered. See https://docs.ansible.com/ansible-core/2.19/reference_appendices/interpreter_discovery.html for more information.
null_resource.webservers (local-exec): [WARNING]: Host 'testserver01' is using the discovered Python interpreter at '/usr/bin/python3.12', but future installation of another Python interpreter could cause a different interpreter to be discovered. See https://docs.ansible.com/ansible-core/2.19/reference_appendices/interpreter_discovery.html for more information.
null_resource.webservers (local-exec): testserver03 | SUCCESS => {
null_resource.webservers (local-exec):     "ansible_facts": {
null_resource.webservers (local-exec):         "discovered_interpreter_python": "/usr/bin/python3.12"
null_resource.webservers (local-exec):     },
null_resource.webservers (local-exec):     "changed": false,
null_resource.webservers (local-exec):     "ping": "pong"
null_resource.webservers (local-exec): }
null_resource.webservers (local-exec): testserver02 | SUCCESS => {
null_resource.webservers (local-exec):     "ansible_facts": {
null_resource.webservers (local-exec):         "discovered_interpreter_python": "/usr/bin/python3.12"
null_resource.webservers (local-exec):     },
null_resource.webservers (local-exec):     "changed": false,
null_resource.webservers (local-exec):     "ping": "pong"
null_resource.webservers (local-exec): }
null_resource.webservers (local-exec): testserver01 | SUCCESS => {
null_resource.webservers (local-exec):     "ansible_facts": {
null_resource.webservers (local-exec):         "discovered_interpreter_python": "/usr/bin/python3.12"
null_resource.webservers (local-exec):     },
null_resource.webservers (local-exec):     "changed": false,
null_resource.webservers (local-exec):     "ping": "pong"
null_resource.webservers (local-exec): }
null_resource.webservers: Creation complete after 14s [id=5720598445132052282]

Apply complete! Resources: 27 added, 0 changed, 0 destroyed.

Outputs:

Ansible client servers got created

3. Install required dependencies — python boto3, amazon.aws collection using ansible-galaxy, aws_cli

$ sudo apt update
$ sudo apt install python3-pip -y
$ sudo apt install python3-boto3

$ aws --version
aws-cli/2.32.6 Python/3.13.9 Linux/6.14.0-1018-aws exe/x86_64.ubuntu.24

configure aws credentials (for demo purpose)

$ aws configure

Install amazon.aws collection

$ ansible-galaxy collection install amazon.aws

4. Configure ansible inventory file — aws_ec2.yml in ansible.cfg

/etc/ansible/aws_ec2.yml

plugin: amazon.aws.aws_ec2

regions:
  - us-east-1

filters:
  instance-state-name: running

keyed_groups:
  - key: tags.Environment
    prefix: env
  - key: tags.Name
    prefix: name
  - key: instance_type
    prefix: type

Inside /etc/ansible/ansible.cfg file add inventory block

[inventory]
enable_plugin = aws_ec2
inventory = /etc/ansible/aws_ec2.yml

5. Execute ansible-inventory commands and Verify inventory using ansible-inventory — graph

$ ansible-inventory -i /etc/ansible/aws_ec2.yml --list

This displays the ec2 resource details under groups in a JSON format

$ ansible-inventory -i /etc/ansible/aws_ec2.yml --graph

@all:
  |--@ungrouped:
  |--@aws_ec2:
  |  |--ec2-44-204-68-83.compute-1.amazonaws.com
  |  |--ec2-34-237-138-220.compute-1.amazonaws.com
  |  |--ec2-44-215-107-193.compute-1.amazonaws.com
  |  |--ec2-3-87-77-234.compute-1.amazonaws.com
  |--@name_Ansible_Vpc_PublicServer_2:
  |  |--ec2-44-204-68-83.compute-1.amazonaws.com
  |--@type_t2_small:
  |  |--ec2-44-204-68-83.compute-1.amazonaws.com
  |  |--ec2-44-215-107-193.compute-1.amazonaws.com
  |  |--ec2-3-87-77-234.compute-1.amazonaws.com
  |--@name_Ansible_Controller:
  |  |--ec2-34-237-138-220.compute-1.amazonaws.com
  |--@type_t2_medium:
  |  |--ec2-34-237-138-220.compute-1.amazonaws.com
  |--@name_Ansible_Vpc_PublicServer_1:
  |  |--ec2-44-215-107-193.compute-1.amazonaws.com
  |--@name_Ansible_Vpc_PublicServer_3:
  |  |--ec2-3-87-77-234.compute-1.amazonaws.com

We can verify the resource details in aws console also

List all instances under group “type_t2_small”

$ ansible -i /etc/ansible/aws_ec2.yml type_t2_small --list-hosts
  hosts (3):
    ec2-44-204-68-83.compute-1.amazonaws.com
    ec2-44-215-107-193.compute-1.amazonaws.com
    ec2-3-87-77-234.compute-1.amazonaws.com

6. Execute ansible-playbook to install nginx on ansible client servers using tags created from ansible inventory.

playbook file:

root@ip-:~/Ansible-Terraform-Testing/playbooks/8.Dyanmic-Inventory# cat dynamic_nginx-jinja2.yml
---
- name: Play for Install Nginx on WebServers
  hosts: "{{ variable_host | default('type_t2_small') }}"
  gather_facts: yes
  become: yes
  become_user: root
  vars:
    #Static Variables
    custom_heading: "Welcome To DevSecOps By Saikiran Pinapathruni"
    custom_para: "We Are Studying Ansible Templates Practically..!!!"
    aws_account: "211125710812"
    friends: ["sai", "avinash", "gnane"]
    #Dynamic variables from Ansible Facts
    todays_date: "{{ ansible_facts['date_time']['date'] }}"
    host_name: "{{ ansible_facts['hostname'] }}"
    fqdn_name: "{{ ansible_facts['fqdn'] }}"
    os_family: "{{ ansible_facts['distribution'] }}"
    os_dest: "{{ ansible_facts['distribution_version'] }}"
    ip_address: "{{ ansible_facts['eth0']['ipv4']['address'] }}"
  tags:
    - InstallNginx
  tasks:
    - name: Run Apt Update
      shell: apt update
      tags:
        - update
    - name: Install Nginx Server
      apt: >
        name=nginx
        state=present
      tags:
        - install
    - name: Copy the index.j2 Template to index destination folder.
      template: >
        src=/root/Ansible-Terraform-Testing/playbooks/5.Jinja/index.j2
        dest=/var/www/html/index.nginx-debian.html
        owner=root
        group=root
        mode=0644
      notify: Restart Nginx
      tags:
        - j2

    - name: Copy the style files to destination folder.
      copy: >
        src=/root/Ansible-Terraform-Testing/playbooks/5.Jinja/style.css
        dest=/var/www/html/style.css
        owner=root
        group=root
        mode=0644
      notify: Restart Nginx
      tags:
        - style

    - name: Copy the javascript files to destination folder.
      copy: >
        src=/root/Ansible-Terraform-Testing/playbooks/5.Jinja/scorekeeper.js
        dest=/var/www/html/scorekeeper.js
        owner=root
        group=root
        mode=0644
      notify: Restart Nginx
      tags:
        - js

  handlers:
    - name: Restart Nginx
      command: service nginx restart

Execute the playbook:

# ansible-playbook -i /etc/ansible/aws_ec2.yml playbooks/8.Dyanmic-Inventory/dynamic_nginx-jinja2.yml

PLAY [Play for Install Nginx on WebServers] *********************************************************************************************************************************************************************************************
[WARNING]: Found variable using reserved name 'tags'.
Origin: <unknown>

tags

TASK [Gathering Facts] ******************************************************************************************************************************************************************************************************************
[WARNING]: Host 'ec2-44-204-68-83.compute-1.amazonaws.com' is using the discovered Python interpreter at '/usr/bin/python3.12', but future installation of another Python interpreter could cause a different interpreter to be discovered. See https://docs.ansible.com/ansible-core/2.19/reference_appendices/interpreter_discovery.html for more information.
ok: [ec2-44-204-68-83.compute-1.amazonaws.com]
[WARNING]: Host 'ec2-3-87-77-234.compute-1.amazonaws.com' is using the discovered Python interpreter at '/usr/bin/python3.12', but future installation of another Python interpreter could cause a different interpreter to be discovered. See https://docs.ansible.com/ansible-core/2.19/reference_appendices/interpreter_discovery.html for more information.
ok: [ec2-3-87-77-234.compute-1.amazonaws.com]
[WARNING]: Host 'ec2-44-215-107-193.compute-1.amazonaws.com' is using the discovered Python interpreter at '/usr/bin/python3.12', but future installation of another Python interpreter could cause a different interpreter to be discovered. See https://docs.ansible.com/ansible-core/2.19/reference_appendices/interpreter_discovery.html for more information.
ok: [ec2-44-215-107-193.compute-1.amazonaws.com]

TASK [Run Apt Update] *******************************************************************************************************************************************************************************************************************
changed: [ec2-3-87-77-234.compute-1.amazonaws.com]
changed: [ec2-44-204-68-83.compute-1.amazonaws.com]
changed: [ec2-44-215-107-193.compute-1.amazonaws.com]

TASK [Install Nginx Server] *************************************************************************************************************************************************************************************************************
ok: [ec2-3-87-77-234.compute-1.amazonaws.com]
ok: [ec2-44-204-68-83.compute-1.amazonaws.com]
ok: [ec2-44-215-107-193.compute-1.amazonaws.com]

TASK [Copy the index.j2 Template to index destination folder.] **************************************************************************************************************************************************************************
changed: [ec2-3-87-77-234.compute-1.amazonaws.com]
changed: [ec2-44-204-68-83.compute-1.amazonaws.com]
changed: [ec2-44-215-107-193.compute-1.amazonaws.com]

TASK [Copy the style files to destination folder.] **************************************************************************************************************************************************************************************
changed: [ec2-3-87-77-234.compute-1.amazonaws.com]
changed: [ec2-44-204-68-83.compute-1.amazonaws.com]
changed: [ec2-44-215-107-193.compute-1.amazonaws.com]

TASK [Copy the javascript files to destination folder.] *********************************************************************************************************************************************************************************
changed: [ec2-44-204-68-83.compute-1.amazonaws.com]
changed: [ec2-3-87-77-234.compute-1.amazonaws.com]
changed: [ec2-44-215-107-193.compute-1.amazonaws.com]

RUNNING HANDLER [Restart Nginx] *********************************************************************************************************************************************************************************************************
changed: [ec2-44-204-68-83.compute-1.amazonaws.com]
changed: [ec2-3-87-77-234.compute-1.amazonaws.com]
changed: [ec2-44-215-107-193.compute-1.amazonaws.com]

PLAY RECAP ******************************************************************************************************************************************************************************************************************************
ec2-3-87-77-234.compute-1.amazonaws.com : ok=7    changed=5    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
ec2-44-204-68-83.compute-1.amazonaws.com : ok=7    changed=5    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
ec2-44-215-107-193.compute-1.amazonaws.com : ok=7    changed=5    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Verify the nginx installation:

$ ansible -i invfile pvt -m shell  -a "nginx -v"
[WARNING]: Host 'testserver03' is using the discovered Python interpreter at '/usr/bin/python3.12', but future installation of another Python interpreter could cause a different interpreter to be discovered. See https://docs.ansible.com/ansible-core/2.19/reference_appendices/interpreter_discovery.html for more information.
testserver03 | CHANGED | rc=0 >>
nginx version: nginx/1.24.0 (Ubuntu)
[WARNING]: Host 'testserver02' is using the discovered Python interpreter at '/usr/bin/python3.12', but future installation of another Python interpreter could cause a different interpreter to be discovered. See https://docs.ansible.com/ansible-core/2.19/reference_appendices/interpreter_discovery.html for more information.
testserver02 | CHANGED | rc=0 >>
nginx version: nginx/1.24.0 (Ubuntu)
[WARNING]: Host 'testserver01' is using the discovered Python interpreter at '/usr/bin/python3.12', but future installation of another Python interpreter could cause a different interpreter to be discovered. See https://docs.ansible.com/ansible-core/2.19/reference_appendices/interpreter_discovery.html for more information.
testserver01 | CHANGED | rc=0 >>
nginx version: nginx/1.24.0 (Ubuntu)

Conclusion

Ansible Dynamic Inventory using the aws_ec2 plugin is a must-have skill for modern DevOps engineers working with AWS.

It enables:

• Fully automated infrastructure discovery
• Reliable deployments
• Tag-based orchestration
• Cloud-native automation

Once this setup is in place, inventory maintenance completely disappears from day-to-day operations.