A case for AWS Systems Manager

Stop using SSH to access your instances: Why AWS Systems Manager is a better way to manage your cloud infrastructure

If you are still managing EC2 instances by logging in one at a time over SSH, running commands manually, and crossing your fingers during the patch window, know there is a better way. As a matter of fact, there are quite a few options, which I will touch on in a bit. But AWS Systems Manager (SSM for short) needs a mention, as it is often overlooked.

AWS Systems Manager (SSM) is a free, built-in AWS service that fundamentally changes how you interact with your server fleet. Here is why it deserves a permanent place in your operations toolkit.

The problem with the traditional way of managing your servers…

Managing even a handful of servers the SSH way is painful. You maintain SSH keys, jump boxes, and security group rules. You run the same command multiple times and manually compare the output. You schedule maintenance windows and hope nothing breaks mid-patch. And every time you need to do something fleet-wide, you either write a fragile shell loop or do it by hand.

Now scale that to 30 servers, or 300, and the problem becomes obvious. Traditional approaches do not scale, and they introduce risk at every step.

What AWS Systems Manager actually does…

SSM is not a single tool: It is a suite of capabilities that work together across your EC2 fleet. The core components relevant to configuration work are:

• Session Manager: Secure browser or CLI-based terminal access to any managed instance, with no inbound ports, no SSH keys, and full audit logging via CloudTrail and CloudWatch.
• Run Command: Execute shell scripts or pre-defined AWS documents across any number of instances simultaneously, targeting by tag, instance ID, or resource group.
• Patch Manager: Automate OS-level patching with scan-before-install workflows, compliance reporting, and maintenance window scheduling.
• Fleet Manager: A unified dashboard showing the health, inventory, and status of every managed node without logging in to any of them.
• Parameter Store: Centralized, encrypted storage for configuration values, secrets, and environment variables — accessible at runtime by your applications.

How about an example…

I am managing a small fleet of 5 EC2 instances to demonstrate SSM’s value proposition.

Step 1: Setting up the example instances

• Set up some EC2 instances, add applicable tags and values(i.e. Environment, Role, etc.) Remember that SSM will use the tags to identify the instances you want to configure
• Make sure the instances have an IAM role with the AmazonSSMManagedInstanceCore policy applied (I named mine instanceRoleSSM but it does not matter if you call it MickeyMouse)

• Have a security group allowing HTTP traffic
• No key pair is required. Remember we don’t require SSH

Step 2: Running some commands on all instances simultaneously

Using Run Command, a single operation checks hostname, uptime, and disk usage across all 5 servers simultaneously. This is work that would otherwise require the same number of SSH sessions and manual note-taking.

• Access the various shell scripts from the run command option in SSM. We will use AWS-RunShellScript here…

• In the Command parameters section, specify the commands you want to run. This will run on all the instances you specify (see next step!)

• Remember to specify the instances you want to have this run on! You do this by using the tags in the Target selection part of this page. Important!! Tags and values are case sensitive

• Now, click on Run at the bottom of the page. The result is a list of the target instances. You can select and view the output, copy it elsewhere or download. You can view information for each instance without having to connect to each individually

Step 3: Using Fleet Manager

All good and well, you say. But I want to get to an individual instance (for whatever reason). You can do so without having to SSH into the device, from the Fleet Manager option.

• Click on Fleet Manager (under Node tools), select an instance and click Connect. This opens a terminal session without you having to SSH

Step 4: Installing a webserver and web page on all instances all at once

• Go back to the Run command, and select AWS-RunShellScript again. In the Command parameters section enter a script to set up a webserver and web page.

• Make sure to apply the tags in order to select the correct instances
• If successful, you will get the following confirmation page. If there are issues you can click on the instance to get details on why something failed

• Check the EC2 instances (remember you set up a security group with port 80 earlier). All 5 instances were available via http:// and showed a message similar to the one below

The nginx web page deployment replaces a repetitive copy-paste workflow with a single targeted command that runs in parallel and reports individual results per instance.

Step 5: Patch Manager

• We are now taking a look at Patch Manager which involves applying a patch policy to multiple instances
• Access Patch Manager under Node Tools, and click Create a Patch Policy (if you have patch policies already, you will bypass that option, but either way the following screen will come up

• You have two options: Scan, and Scan and Install. It is a best practice to always run a scan first
• You can save output to an S3 bucket you specify
• Don’t forget to apply tags here as well, to make sure you will be patching the correct set of instances.
• Click Run. If this is the first time running Patch Manager you will get a pop-up letting you know that.
• Be aware that you will be taken to the Quick Setup page where you can see the Patch Manager results. If you get lost, you can always find this under Change Management Tools > Quick Setup

• Note that you that it will also show inactive servers if they fall under your tag labeling criteria
• Under Quick Setup you can find additional information. The Associations tab shows you success or failure of your Patch Manager runs, and gives you the reason why. In the example below one run failed because I forgot to specify an S3 bucket to store the results (this is optional btw)

Step 6: Now to actually applying a patch

• Go to Patch Manager. Select the patch needed from the Patches tab. You will get a Patch description for the selected item
• You can either apply the patch immediately (Patch now) or create a patch policy. You have the option to allow for system reboots if needed

Your company will have policies on the patching process, schedule, CAB reviews, etc. Also, if things do not go well with a particular patch you can rollback using AWS Systems Manager, or you can use a snapshot you took of the environment before you applied the patch in question. The latter is another best practice.

In closing: Patch Manager adds another layer: it scans the fleet for missing patches before committing to installation, giving visibility and control that traditional cronjob-based patching simply cannot match.

What did this example demonstrate?

• We did not require any inbound ports: Session Manager operates over HTTPS/443 outbound from the agent, eliminating the attack surface of open SSH ports entirely.
• Tag-based targeting: Target any subset of your fleet by tag — no manual instance ID lists to maintain.
• Parallel execution with per-instance results: Run Command shows output from each server individually, making it easy to spot anomalies across a fleet.
• Full audit insight: Every session and command execution is logged. This is invaluable for compliance, incident response, and change management.
• No additional cost for core features: Session Manager, Run Command, Patch Manager, and Fleet Manager are all free for EC2 instances.
• No bastion hosts: Eliminates the operational overhead and security risk of maintaining jump boxes.

But that is not all!

The example is not doing the SSM capabilities justice — there is so much more under the hood! Once you have SSM set up, the same infrastructure supports a much broader set of operational tasks:

• Automated compliance remediation: Use State Manager to continuously enforce configurations — ensuring the SSM agent is running, specific packages are installed, or file permissions are correct.
• Secrets management: Parameter Store (with SecureString) replaces hardcoded credentials in scripts and application configs.
• Hybrid infrastructure: SSM supports on-premises servers and VMs via the Hybrid Activations feature, so the same tooling works across cloud and data center.
• Automation runbooks: SSM Automation documents can orchestrate multi-step workflows — for example, snapshot an EBS volume, apply a patch, and reboot, all as a single automated sequence.
• Inventory collection: Automatically collect software inventory, network configuration, and file metadata from every managed node.

How to get started with SSM

If your EC2 instances are running Amazon Linux 2, Amazon Linux 2023, or Ubuntu 20.04+, the SSM agent is already installed. Attach the AmazonSSMManagedInstanceCore IAM policy to the instance role and the instance appears in Fleet Manager within minutes. That is the entire setup for the core features.

For teams moving away from SSH-based workflows, AWS Systems Manager offers a more secure, more scalable, and more auditable way to operate cloud infrastructure.

Other options…

OK, I did say that AWS Systems Manager is but one of several approaches to configuration management. There is Chef, Puppet, Ansible, and many more…

I have no experience with Chef and have only limited exposure to Puppet. I have, however, worked with Ansible. Ansible is a low hurdle to jump if you are comfortable with YAML. In this follow-up blog post: https://medium.com/@gkinthaert/aws-systems-manager-vs-ansible-8cdd386914eb you can see me apply Ansible to this very same example, and document some observations on the experience.