Real-Time Docker Metrics with AWS CloudWatch: Dynamic Integration for Containers

Monitoring Docker containers is crucial for maintaining performance, diagnosing issues, and ensuring resource utilization is optimized. While AWS CloudWatch offers robust monitoring capabilities, out-of-the-box Docker doesn’t automatically send its resource metrics (like CPU, memory, I/O) to CloudWatch.

In this guide, you’ll learn how to dynamically send Docker container metrics — gathered using docker stats — to AWS CloudWatch Metrics automatically every time a container starts.

🧠 Why Send Docker Stats to CloudWatch?

By pushing container-level stats to CloudWatch:

• You can visualize resource usage per container in the AWS Console.
• Create alarms and notifications for high CPU, memory usage, or container crashes.
• Get centralized, long-term metrics storage even for short-lived containers.

🔧 What You’ll Build

You’ll implement a setup where:

• Each time a Docker container starts, a background process (or sidecar) collects its stats.
• The stats are parsed and pushed to AWS CloudWatch Metrics using the AWS CLI or SDK.
• If necessary, CloudWatch namespaces and custom metrics are created dynamically.

🛠️ Prerequisites

• AWS CLI installed and configured with sufficient permissions (cloudwatch:PutMetricData)
• AWS CloudWatch agent installed and running
• Docker installed and running
• IAM Role or User with access to CloudWatch
• Basic scripting knowledge (we’ll use Bash and AWS CLI in this example)

🚀 Step-by-Step Implementation

1. Create a Bash Script to Push Metrics

#!/bin/bash

# AWS CloudWatch namespace

TOKEN=$(curl -sX PUT "http://169.254.169.254/latest/api/token" \
  -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")

IDENTITY=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" \
  http://169.254.169.254/latest/dynamic/instance-identity/document)

REGION=$(echo "$IDENTITY" | jq -r .region)
INSTANCE_ID=$(echo "$IDENTITY" | jq -r .instanceId)
NAMESPACE="your_namespace" # Replace with your CloudWatch namespace

# Get list of container names
CONTAINERS=$(docker ps --format '{{.Names}}')

for CONTAINER in $CONTAINERS; do
    # Extract multiple metrics
    METRICS=$(docker stats --no-stream --format "{{.CPUPerc}} {{.MemUsage}} {{.NetIO}} {{.BlockIO}}" $CONTAINER)

    CPU=$(echo $METRICS | awk '{print $1}' | sed 's/%//')

    MEM_USED=$(echo $METRICS | awk '{print $2}' | sed 's/[^0-9\.]//g')
    MEM_UNIT=$(echo $METRICS | awk '{print $2}' | sed 's/[0-9\.]//g')

    NET_IO=$(echo $METRICS | awk '{print $3}')
    NET_RX=$(echo $NET_IO | cut -d'/' -f1 | sed 's/[^0-9\.]//g')
    NET_TX=$(echo $NET_IO | cut -d'/' -f2 | sed 's/[^0-9\.]//g')

    BLOCK_IO=$(echo $METRICS | awk '{print $4}')
    BLK_READ=$(echo $BLOCK_IO | cut -d'/' -f1 | sed 's/[^0-9\.]//g')
    BLK_WRITE=$(echo $BLOCK_IO | cut -d'/' -f2 | sed 's/[^0-9\.]//g')

    # Normalize memory to MB
    case "$MEM_UNIT" in
        kB) MEM_USED=$(awk "BEGIN {print $MEM_USED / 1024}") ;;
        GiB) MEM_USED=$(awk "BEGIN {print $MEM_USED * 1024}") ;;
        MiB) ;; # Already in MB
        B) MEM_USED=$(awk "BEGIN {print $MEM_USED / 1024 / 1024}") ;;
    esac

    # Push to CloudWatch
    if [[ "$CPU" =~ ^[0-9.]+$ ]]; then
    aws cloudwatch put-metric-data --namespace "$NAMESPACE" --region "$REGION" --metric-name "CPUUtilization" \
        --dimensions ContainerName=$CONTAINER,InstanceId=$INSTANCE_ID --value "$CPU" --unit Percent
    fi

    if [[ "$MEM_USED" =~ ^[0-9.]+$ ]]; then
    aws cloudwatch put-metric-data --namespace "$NAMESPACE" --region "$REGION" --metric-name "MemoryUsageMB" \
        --dimensions ContainerName=$CONTAINER,InstanceId=$INSTANCE_ID --value "$MEM_USED" --unit Megabytes
    fi

    if [[ "$NET_RX" =~ ^[0-9.]+$ ]]; then
    aws cloudwatch put-metric-data --namespace "$NAMESPACE" --region "$REGION" --metric-name "NetworkRxKB" \
        --dimensions ContainerName=$CONTAINER,InstanceId=$INSTANCE_ID --value "$NET_RX" --unit Kilobytes
    fi

    if [[ "$NET_TX" =~ ^[0-9.]+$ ]]; then
    aws cloudwatch put-metric-data --namespace "$NAMESPACE" --region "$REGION" --metric-name "NetworkTxKB" \
        --dimensions ContainerName=$CONTAINER,InstanceId=$INSTANCE_ID --value "$NET_TX" --unit Kilobytes
    fi

    aws cloudwatch put-metric-data --namespace "$NAMESPACE" --region "$REGION" --metric-name "BlockReadKB" \
        --dimensions ContainerName=$CONTAINER,InstanceId=$INSTANCE_ID --value "$BLK_READ" --unit Kilobytes

    aws cloudwatch put-metric-data --namespace "$NAMESPACE" --region "$REGION" --metric-name "BlockWriteKB" \
        --dimensions ContainerName=$CONTAINER,InstanceId=$INSTANCE_ID --value "$BLK_WRITE" --unit Kilobytes
done

Save it as docker-to-cloudwatch.sh

2. Create a systemd service & timer

[Unit]
Description=Push Docker Container Stats to CloudWatch
After=docker.service network.target

[Service]
ExecStart=/usr/local/bin/docker-to-cloudwatch.sh
User=root
Type=oneshot

[Install]
WantedBy=multi-user.target

Save it as docker-metrics.service

[Unit]
Description=Run Docker Metrics Script Every Minute

[Timer]
OnBootSec=1min
OnUnitActiveSec=1min
Unit=docker-metrics.service

[Install]
WantedBy=timers.target

Save it as docker-metrics.timer

systemctl daemon-reexec
systemctl daemon-reload
systemctl enable docker-metrics.timer
systemctl start docker-metrics.timer

#Check logs of the service
journalctl -u docker-metrics.service

🧩 Final Thoughts

While AWS doesn’t natively track Docker stats without deeper integration (like ECS/EKS), this solution allows you to quickly and flexibly stream Docker stats to CloudWatch with minimal setup.

For production-grade environments, consider extending this with:

• Docker labels to auto-tag metrics
• CloudWatch alarms on thresholds
• Visualization dashboards in CloudWatch