19 Tips to Optimize Ansible Performance: Mastering Performance Optimization for Smarter Automation

Everything you need to know to speed up your Ansible playbooks!

Not a Medium member? Read here!

Introduction

If you’re looking to enhance the performance and efficiency of your Ansible playbooks, you’re in the right place! In the world of IT automation, even the smallest tweaks can greatly improve execution speed and resource use. Whether you’re handling cloud setups, configuring servers, or orchestrating intricate deployments, optimizing your Ansible playbooks is key to achieving seamless operations.

In this article, we will explore 19 actionable tips and tricks that promise to drastically boost the performance of your playbooks.

Here is a rough overview of what we will talk about:

• Callback plugins
• Parallel task execution
• Asynchronous tasks (async and poll)
• Optimizing fact gathering (including gather_subset)
• Caching
• smart fact gathering
• Playbook execution strategies for performance optimization
• Tips on optimizing module execution
• Optimizing SSH connections
• Using inventories in the right way

I tried to make this as comprehensive as I could at the moment of writing. Happy reading (and configuring)!

Callback plugins

1. Leverage callback Plugins

Callback plugins in Ansible allow you to add specific behaviors in response to designated events. They can format task output, log playbook events, send notifications, and much more.

Among these, the following plugins are particularly useful for measuring task execution time and overall performance:

⤬ timer (ansible.posix.timer)

• Displays a summary of task execution time in the play statistics.

⤬ profile_tasks (ansible.posix.profile_tasks )

• Reports timing information for individual tasks.

⤬ profile_roles (ansible.posix.profile_roles)

• Provides timing related to Ansible roles.

To enable these plugins, modify the Ansible configuration file, typically located at /etc/ansible/ansible.cfg, as follows:

[defaults]
callbacks_enabled = timer, profile_tasks, profile_roles

Once enabled, you can execute a playbook to see the plugin functionality in action.

ansible-playbook -i inventory.ini display_facts.yaml

Here’s a sample playbook, display_facts.yaml, to gather and display facts:

# display_facts.yaml
- name: Gather facts about the nodes
  hosts: all
  tasks:
    - name: Gather facts
      ansible.builtin.setup:
    - name: Display gathered facts
      ansible.builtin.debug:
        var: ansible_facts

At the end of the playbook output, you will see a summary of execution time for each task in particular and for the playbook in total, like this:

Documentation:

• https://docs.ansible.com/ansible/latest/plugins/callback.html
• https://docs.ansible.com/ansible/latest/reference_appendices/config.html#callbacks-enabled
• https://docs.ansible.com/ansible/2.9/plugins/callback/timer.html
• https://docs.ansible.com/ansible/latest/collections/ansible/posix/profile_tasks_callback.html
• https://docs.ansible.com/ansible/2.9/plugins/callback/profile_roles.html

Speeding up task execution

2. Use parallel execution

Ansible executes tasks in parallel by default. The degree of parallelism can be controlled with the forks parameter.

The forks parameter determines how many simultaneous connections Ansible can make to execute tasks across managed nodes.

• The default value is set to 5, meaning that Ansible will execute tasks on up to 5 hosts concurrently, and after finishing, move on to the next batch of 5 hosts.

Increasing the value of the forks parameter boosts the speed at which playbooks are executed across managed nodes, thanks to parallelism.

To change the value, update the Ansible configuration file (/etc/ansible/ansible.cfg):

[defaults]
forks = 20

Alternatively, you can set the forks parameter on a per-command basis using the -f or --forks option:

ansible-playbook playbook.yaml --forks 20

Note: Be cautious with very high values. Every fork increases the number of simultaneous SSH connections, and too many of these can put a strain on your Ansible controller, potentially overwhelming it.

Documentation:

• https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_strategies.html#setting-the-number-of-forks

3. Execute tasks asynchronously

By default, tasks in Ansible are executed synchronously, which means that Ansible waits for all targeted hosts to complete a task before proceeding to the next one. This approach can severely hinder execution speed — especially with long-running tasks.

To boost performance, configure Ansible to run tasks asynchronously using the async and poll parameters:

⤬ async

• The async parameter sets the timeout for a task in seconds. For example, a value of 1800 allows a task to execute for 30 minutes before timing out.

⤬ poll

• The poll parameter how frequently Ansible checks for the completion of an asynchronous task.
• The default value is 15. This means that Anisble will check a task for completion each 15 seconds.
• Setting poll to 0 disables checks completely — allowing for immediate transition to the next task.

poll = 0 in combination with async allows Ansible to kick off tasks asynchronously and instantly move on to subsequent tasks. Tasks will run until they either complete or reach the async-specified timeout, without any checks taking place during execution.

Here is an example:

- name: Execute a long-running task in the background
  command: ./script.sh
  async: 1800 # set timeout to 30 minutes
  poll: 0     # disable task completion checks

You can also set the async and poll parameters through the -B (--background) and -P (--poll) command-options to the ansible command, although only on per-task basis. For example:

ansible -B 1800 -P 0 "/path/to/long_running_task"

Documentation:

• https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_async.html

Dealing with facts

4. Disable fact gathering

By default, Ansible gathers facts about each host before executing tasks. This adds certain overhead. If you don’t use facts, disable their collection by setting the gather_facts directive to false in your playbook:

- hosts: all
  gather_facts: false
  tasks:
    # ...

This is especially relevant for small playbooks, where fact gathering can take up nearly half of the total execution time. Below are some statistics showcasing the performance of a simple playbook with fact gathering turned on and off:

• With fact gathering:

• Without fact gathering:

5. Conditional fact gathering

Rather than entirely disabling fact gathering, consider using conditional gathering based on specific host criteria.

For example, gather facts only on a host identified by a specific name. Subsequently, perform tasks that utilize this information solely on that particular host:

- hosts: all
  tasks:
    - name: Gather facts conditionally
      setup:
      when: ansible_hostname == "specific_host"

    - name: Use variables if facts were gathered
      debug:
        var: ansible_distribution
      when: ansible_hostname == "specific_host"

Note: The ansible_hostname itself is a host-specific inventory variable, not a fact. Be careful not to use a fact a basis to determine if you should gather that fact — this will obviously fail.

6. Gather only necessary facts with fact subsets

If only certain facts are needed, limit data collection using the [gather_subset](https://docs.ansible.com/ansible/latest/collections/ansible/builtin/setup_module.html#parameter-gather_subset) parameter. By default, it is to ["all"], but you can specify a list of fact subsets to gather exactly what you need.

For instance, if you want to collect information solely about the network and processor of each host, you can set gather_subset to ['network', 'processor']:

- hosts: all
  gather_facts: true
  gather_subset:
    - "network"
    - "processor"
  tasks:
    name: Install Nginx
    # ...

A full list of options can be found in the official documentation:

• https://docs.ansible.com/ansible/latest/collections/ansible/builtin/setup_module.html#parameter-gather_subse

7. Implement fact caching

By default, Ansible re-gathers facts during each playbook execution, regardless of changes. This can substantially hamper performance.

To mitigate this, configure a caching mechanism to temporary store these facts, avoiding repeated queries.

Caching options include JSON files and Redis.

JSON files:

• This method stores cached facts as JSON files locally on the Ansible controller. Here is how to configure it in the ansible.cfg file:

[defaults]
fact_caching = jsonfile
fact_caching_connection = /tmp/ansible_fact_cache
fact_caching_timeout = 86400 # cache for 24 hours

⤬ fact_caching

• Specifies which cache plugin to use.

⤬ fact_caching_connection

• Defines connection or path information for the cache plugin.

⤬ fact_caching_timeout

• Specifies expiration time for which cached facts are considered valid.

Once facts are gathered for the first time, they will appear in JSON files located in the directory specified by the fact_caching_connection directive. Each host will have its own subdirectory containing its cached facts:

ls /tmp/ansible_fact_cache

ansible_node

cat /tmp/ansible_fact_cache/ansible_node

Redis:

This alternative method offers a more robust caching solution, although requiring additional installation of Redis.

First, here is how to install Redis:

# Ubuntu/Debian
sudo apt-get install lsb-release curl gpg
curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg
sudo chmod 644 /usr/share/keyrings/redis-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/redis.list
sudo apt-get update
sudo apt-get install redis

sudo yum install redis # RHEL/CentOS

sudo pacman -S redis   # Arch/Manjaro

After installation, enable and start the Redis service:

sudo systemctl enable redis # or redis-server

sudo systemctl start redis  # or redis-server

Then modify your Ansible configuration to include the following:

fact_caching = redis
fact_caching_timeout = 86400 # cache for 24 hours
fact_caching_connection = localhost:6379:0

Documentation:

• https://docs.ansible.com/ansible/latest/plugins/cache.html
• https://docs.ansible.com/ansible/latest/reference_appendices/config.html#cache-plugin
• https://docs.ansible.com/ansible/latest/reference_appendices/config.htm
• https://docs.ansible.com/ansible/latest/reference_appendices/config.html#cache-plugin-timeout

8. Use smart fact gathering

Introduced in Ansible version 2.7, Smart Fact Gathering optimizes Ansible fact collection by only gathering facts when strictly necessary: Ansible first checks if the facts are already available from previous plays, and if yes, skips querying them again.

This is in contrast to the default behavior, when Ansible blindly gathers facts at the beginning of every play.

To enable Smart Fact Gathering, update your ansible.cfg:

[defaults]
gathering = smart

Documentation:

• https://docs.ansible.com/ansible/latest/reference_appendices/config.html#default-gatheringsmart

Optimizing playbooks and modules

9. Choose effective execution strategies

Ansible offers different playbook execution strategies that determine how tasks should be executed across multiple hosts.

The default linear strategy executes tasks sequentially for all hosts (using forks, as mentioned above). A more effective alternative is the free strategy. It allows the next task to run on a host as soon as the current one finishes, with each host being independent of others. This can significantly enhance performance.

The strategy can be set in playbooks with the strategy directive:

- hosts: all
  strategy: free
  tasks:
  # ...

You can also set it as the default in the ansible.cfg file:

[defaults]
strategy = free

Documentation:

• https://docs.ansible.com/ansible/2.9/user_guide/playbooks_strategies.html

Optimizing playbooks and modules

9. Choose effective execution strategies

Ansible offers different playbook execution strategies that determine how tasks should be executed across multiple hosts.

The default linear strategy executes tasks sequentially for all hosts (using forks, as mentioned above). A more effective alternative is the free strategy. It allows the next task to run on a host as soon as the current one finishes, with each host being independent of others. This can significantly enhance performance.

The strategy can be set in playbooks with the strategy directive:

- hosts: all
  strategy: free
  tasks:
  # ...

You can also set it as the default in the ansible.cfg file:

[defaults]
strategy = free

Documentation:

• https://docs.ansible.com/ansible/2.9/user_guide/playbooks_strategies.html

10. Package installations and loops

When utilizing the apt module to install packages, the way you structure the task can significantly impact performance.

When using the apt module to install packages, the way you structure the task can have a substantial impact on performance.

By default, if you pass a list of package name for installation using loop or with_items, Ansible launches the package manager separately for each iteration of the loop. This means that if you're installing 10 packages, apt will run 10 times — once for each package.

In other words, the following playbook:

- name: Install packages
  apt:
    name: "{{ item }}"
    state: present
  loop:
    - less
    - rsync
    - firewalld
    # ...

will result in this sequence of commands:

apt install less ; \
apt install rsync ; \
apt install firewalld

Each execution of apt involves overhead due to the system needing to invoke the package manager program multiple times.

To optimize this, starting from version 2.3 and later, you can pass a list of packages directly to the name parameter of the apt module. This allows the apt manager to process all package installations in a single command.

This is how it looks like:

- name: Install packages
  apt: 
    state: present
    name:
      - less
      - rsync
      - firewalld
      # ...

The above is equivalent to the following command:

apt install less rsync firewalld # ...

11. Choose modules wisely

Opt for native Ansible modules instead of shell commands whenever possible. For example, instead of executing an installation command through a shell:

- name: Install Apache using shell command
  shell: apt install -y apache2

Leverage the apt module, which interacts directly with the package manager:

- name: Install Apache using Ansible module
  apt:
    name: apache2
    state: present

Native Ansible modules offer optimized performance, compared to their command equivalents. In contrast, using a shell command leads to overhead associated with spawning a new shell process and invoking the apt process from there.

You can find the full list of Ansible modules here:

• https://docs.ansible.com/ansible/2.9/modules/list_of_all_modules.html

12. Use the synchronize module for file transfers

When transferring multiple files between the Ansible controller and managed nodes, prefer the synchronize module over the copy module:

• **synchronize**:

- name: Transfer application data
  synchronize:
    mode: push
    src: source_directory/         # Source on the control machine
    dest: /dest_directory          # Destination on the target machine

• **copy**:

- name: Copy application data
  copy:
    src: "{{ item }}"
    dest: /dest_directory
  loop:
    - source_directory/file1.txt
    - source_directory/file2.txt
    - source_directory/file3.txt
    # Additional files...

The reason why synchronize is more efficient than copy lies in how these modules manage SSH connections with managed nodes:

• When using copy loop, Ansible opens a new SSH session for each file transfer. This can lead to network overhead and latency, especially when transferring multiple files.
• At the same time, synchronize uses rsync under the hood, which means it typically opens a single SSH connection per synchronization, and deals with recursive copying of entire directories with ease.

Documentation:

• https://docs.ansible.com/ansible/latest/collections/ansible/posix/synchronize_module.html

13. Limit variables in playbooks

Excessive use of variables, especially when defined in multiple places (like playbooks, role defaults, inventory, etc.), can lead to increased resolution time as Ansible determines which variable to use, and performance degradation as a consequence.

Aim to use fewer variables and manage their scope wisely to optimize your playbooks.

14. Avoid using loop or with_items for large lists

When using loops (e.g., with_items or loop) with extensive lists, performance may suffer. Seek alternatives or optimization strategies, such as employing conditionals, dynamically including tasks, or using Jinja2 filters to reduce unnecessary repetitions.

15. Keep modules updated

Regularly update to the latest compatible version of Ansible. New releases often contain performance optimizations and feature enhancements that can significantly speed up execution.

Optimizing SSH

16. Use SSH connection multiplexing

To execute each task, Ansible creates a new SSH session with each host. By default, this implies establishing a separate TCP connection per each new session.

For example, in a playbook consisting of 10 tasks executed across 10 hosts, you would end up with 100 TCP connections — one for each task and each host. This repetitive authentication slows down playbooks considerably.

The solution? Leverage SSH multiplexing supported by OpenSSH.

OpenSSH Multiplexing allows multiple SSH sessions to reuse an existing TCP connection.

In the above example, with SSH multiplexing enabled, the number of TCP connections drops dramatically to just 10 — one for each host.

How it works:

• When connecting to a remote host, a master SSH connection is established first. Ansible stores the socket for this connection in the directory we will configure.
• Subsequent SSH sessions to the same host reuse the same existing master connection instead of initiating a new TCP handshake each time.

Configuration steps:

1. Create a directory for TCP sockets

We first need to create a directory where TCP sockets for master connections to each host will be stored. A TCP socket is a file that acts as an endpoint for sending and receiving data across the network; reusing the same TCP connection will mean reusing the same TCP socket.

We will create ~/.ssh/ansible:

mkdir ~/.ssh/ansible

2. Modify Ansible Configuration

[ssh_connection]
ssh_args = -o ControlMaster=auto -o ControlPersist=3600 -o ControlPath='~/.ssh/ansible/ansible-%r@%h-%p'

⤬ ControlMaster=auto

• Upon initiating a new SSH session, automatically sets up a master connection. This connection will be used by future SSH sessions.

⤬ ControlPersist=3600

• Keeps the master connection open in the background for 3600 seconds (1 hour). For indefinite persistence, change to ControlPersist yes.

⤬ ControlPath='~/.ssh/ansible-%%r@%%h:%%p'

• Defines the path to the control socket file for the master TCP connection. Here, we use the same directory that was created before.

⤬ In the socket file name variables mean:

• %r specifies the remote user;
• %h is the remote hostname;
• $p is the remote SSH port.

After configuration, when you run a playbook, you will see TCP connection sockets created in ~/.ssh/ansible — one for each host:

ls -1 ~/.ssh/ansible

ansible-ssh-ansible@172.0.17.2:22

Note: Be mindful of the maximum number of SSH sessions that can reuse the same TCP socket. This limit is determined by the MaxSessions directive in the SSH server configuration of your Ansible-managed nodes.

17. Use SSH pipelining

Pipelining in Ansible is a feature that can significantly enhance the performance of your playbooks by reducing the number of SSH connections needed during task execution.

By default, Ansible works by creating a new SSH connection for each task, transferring the module code as a file, executing it, reporting the results back to the control machine, and then closing the connection.

SSH pipelining allows multiple commands or tasks to be sent over an SSH session in a single request instead of sending them one at a time.

SSH pipelining is useful in scenarios involving scripts or automation tasks where a series of commands need to be executed in succession, without waiting for each to complete before sending the next. Just as with Ansible.

To enable pipelining, you need to set the pipelining = True in the Ansible configuration file:

[ssh_connection]
pipelining = True

Documentation:

• https://docs.ansible.com/ansible/latest/reference_appendices/config.html#ansible-pipelining

18. Handle unreachable hosts gracefully

When using static inventories, unreachable hosts can cause delays due to multiple connection retries, which often stalls execution.

To optimize this, set the ConnectTimeout directive in your Ansible configuration file to control SSH connection timeout (in seconds) more effectively:

[ssh_connection]
ssh_args = -o ConnectTimeout=10

⤬ ConnectTimeout=10

• Sets the timeout for attempts to establish a connection with a target node to 10 seconds. If the host remains unreachable for this time, Ansible gives up any further attempts to contact it.

Using inventories in the right way

19. Use dynamic inventories

Instead of relying on large static inventories that may include unnecessary hosts, leverage dynamic inventories whenever possible. This allows you to create inventories that reflect the current state of the infrastructure and reduce unnecessary processing.

Conclusion

Summary:

1. Use callback plugins to measure task execution time.
2. Use parallel task execution (forks).
3. Execute tasks asynchronously (async and poll=0).
4. Disable fact gathering when not needed (gather_facts: false).
5. Gather facts conditionally (when).
6. Gather only necessary facts (gather_subset).
7. Implement fact caching (JSON files or Redis).
8. Use smart fact gathering (gathering = smart).
9. Use the free playbook strategy (strategy = free).
10. Pass list of packages to apt directly.
11. Prefer native Ansible modules over their command equivalents.
12. Use synchronize instead of copy for multiple file transfers.
13. Limit variables in playbooks.
14. Avoid using loop or with_items for large lists.
15. Keep Ansible updated.
16. Use SSH connection multiplexing.
17. Use SSH pipelining.
18. Set connection timeout to handle unreachable hosts (ConnectTimeout).
19. Use dynamic inventories.

Optimizing your Ansible playbooks requires continuous learning and adaptation. Regularly review your playbooks for potential improvement, and keep yourself informed about the newest features and the best practices.

So, why wait? Start optimizing your Ansible playbooks today, and watch how these enhancements can transform your automation journey into a smoother, more efficient experience!