A day as a DevOps Docker Engineer: afternoon tea

Part III of III: Swarm, Secrets, Resource Management, Monitoring & Backup

Introduction

This episode goes over production-grade operations: orchestration, security, resource governance, and disaster recovery, and I will look at Docker Swarm for clustering, Secrets for sensitive data, resource limits for stability, and monitoring tools for visibility.

Docker Swarm

Task 11: Initialize Swarm

Docker Swarm is Docker’s native clustering and orchestration mode. A Swarm consists of manager nodes (which schedule and manage services) and worker nodes (which run the actual containers). (FYI — For production, you want at least three managers for high availability)

Run the docker command docker info again. You will see the swarm status, and it will probably state inactive

docker swarm init

docker info (now shows a new status)

When your run the initializing command, your node will be set up as the default manager:

When you run docker node ls you will see your node set up as the leader:

Let’s take the opportunity to create a worker node (I suggest an Ubuntu 2GB EC2 instance)! Install Docker on the worker node, as such:

sudo apt update
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
sudo usermod -aG docker ubuntu
newgrp docker

Make sure the security group for the instances in the swarm allows Port 2377 TCP (Cluster management), Port 7946 TCP + UDP (Node communication), and Port 4789 UDP (Overlay network)

Next, grab the docker swarm join command that was generated on the manager node when you initialized it! Run this command on the worker node. When you again run docker node ls (ONLY AVAILABLE ON THE MANAGER NODE) you will see the worker listed!

💡 Why This Matters: Swarm mode transforms the Docker CLI into an orchestrator. Services (not containers) become the unit of deployment. Swarm decides which node runs each task, redistributes them if a node fails, and maintains your desired replica count automatically.

💡 Important note on nodes: If you have no worker nodes, then the service will run on the manager (leader) node by default. However, as soon as you have a worker node, then best practice would be to have the services run on the worker(s) only. That leaves the manager node to focus on orchestrating only. You do that by setting the manager node’s availabiltiy to ‘drain’, with the command below:

docker node update --availability drain <NODE-ID-OR-HOSTNAME>

When your manager is in this drain state, and you are wondering why you can’t find a running container when running docker ps, then try running it on the worker. I bet you dinner you will find it there, on the worker node(s)

Additional: Management type commands such as docker service ls, will only run on the manager node!

Task 12: Deploy Application as a Swarm Service

In Swarm, you deploy services rather than containers. A service is a desired state declaration: run this image, with this many replicas, on this port. Swarm keeps that state true even as nodes come and go.

docker service ls

Task 13: Scale Services and Verify

One of Swarm’s headline features: scaling with a single command.

docker service scale demoservice=4

You can see that the service was scaled to 4 replicas.

docker service ls
docker service ps demoservice
docker service inspect demoservice
docker service logs demoservice (gives you additional insights when troubleshooting)

output from docker service inspect demoservice shows the requested replicas:

💡 Use Case: Horizontal scaling is why you use an orchestrator. Under load, you scale up: docker service scale webapp=10. When the load drops, you scale back down. This is the operational pattern that makes container orchestration attractive.

Secrets and Configs

Task 14: Docker Secrets

Hardcoding passwords, API keys, and connection strings into images or environment variables is a serious security risk. Docker Secrets solve this properly: sensitive values are encrypted at rest, only decrypted in memory inside the container, and never visible in docker inspect output or image layers.

Create a secret from a file:

echo "secret-database-pw" > db-password.txt
docker secret create db_password db-password.txt
docker secret ls

The .txt file can now be deleted — the secret lives in Swarm’s encrypted store.

Attach the secret to a service:

docker service create --name secret-service --secret db_password nginx

Access it inside the running container (remember this will be on the worker node where the task runs):

docker exec -it [container_id] bash (access the container)
cat /run/secrets/db_password  (read the secret on the container)

💡 Security Note: Know (and check for yourself) that docker inspect [container_id] does NOT reveal the secret value. Only processes running inside the container with appropriate access can read /run/secrets/. (This is what we just did by logging into the container). This is a fundamental improvement over passing passwords as -e environment variables, which are visible in docker inspect output.

Task 15: Docker Configs

Docker Configs work similarly to Secrets, but for non-sensitive configuration data — application settings, nginx.conf files, welcome messages — that you want to manage centrally rather than baking into your image.

nano app.conf   # create the config file
docker config create app_config app.conf
docker config ls

Unlike secrets, you can inspect config content (it’s not encrypted):

docker inspect app_config

Now, how do you attach the Config to a service?

docker service create --name configdemo --config app_config nginx

Now, access the config file inside the container — it appears at /[config_name]:

💡 Use Case: Configs let you deploy the same image to multiple environments with different configuration. Your nginx config in staging might log at debug level; production logs at warn. Same image, different config attached at service creation time. No image rebuild required!

Resource Management

Tasks 16: CPU and Memory Limits

Without resource limits, a misbehaving container can starve every other container on the host. Setting CPU and memory limits is basic operational hygiene — it turns Docker into a well-governed multi-tenant environment rather than a free-for-all.

docker service create --name limitdemo -p 8080:80 --limit-cpu 0.4 --limit-memory 100m nginx

and then verify that the limits are applied!

docker service inspect limitdemo

The limits appear in the Resources section:

• 0.4 CPU = 400,000,000 NanoCPUs (the kernel’s internal representation)
• 100MB memory = 104,857,600 bytes

💡 Best Practice: Set both — limit-cpu and — limit-memory, but also consider — reserve-cpu and — reserve-memory. Limits cap maximum usage; reservations guarantee minimums. Without reservations, Swarm might schedule too many services on one node, and they’ll all fight for resources.

💡 Use Case: In a shared cluster running both high-priority API services and lower-priority batch jobs, resource limits ensure that a runaway batch job cannot starve the API. Start conservative with limits, monitor actual usage, and adjust based on real data.

Monitoring and Troubleshooting

Task 17: Monitor Running Containers

You can’t fix what you can’t see. Docker provides several built-in monitoring tools that are useful for quick spot-checks and diagnosing live issues.

docker stats [container id] (real time stats for CPU, memory, network, and I/O)
docker events (Live event stream — shows start, stop, exec, and other container lifecycle events)
docker top [container id] (Active processes inside a container (like ps aux, but from the host's perspective)

The docker top output shows three key columns:

• UID: the system user account running the process inside the container
• PID: the process ID as seen by the host Linux kernel
• PPID: the parent process ID — which process spawned this one

💡 Use Case: When a container is consuming unexpectedly high CPU, docker stats gives you the first indication. docker top then shows which process inside the container is responsible. docker events helps you understand the sequence of events that led to the problem.

Task 18: Generate and analyze logs

Container logs are the first place to look when something isn’t working.

docker ps  # get container id
docker logs [container id]
docker logs -f [container id]

💡 Best Practice: Configure your containers to log to stdout/stderr (not to files inside the container). Docker captures stdout/stderr automatically with docker logs. If you log to files inside the container, you need to exec in or set up volume mounts to read them — much more painful.

Task 19: Service Update and Rollback

Zero-downtime updates and instant rollbacks are among Swarm’s most operationally valuable features. This is the workflow that lets you deploy with confidence.

example: let’s start with creating a service with 3 replicas, with a nginx:1.25 image

docker service create --name corporate-website --replicas 3 nginx:1.25

Next, update to a newer version of the image

docker service update --image nginx:latest corporate-website

Watch the rolling update in progress:

docker service ps corporate-website 

But, something went wrong! Abort! Abort! This is when the rollback command saves the day!

docker service rollback corporate-website

💡 Best Practice: Always deploy services with a specific version tag rather than latest, so rollbacks are predictable. Configure — update-parallelism 1 and — update-delay 10s for production services — this rolls out changes one replica at a time with a delay between each, giving you time to detect problems before all replicas are updated.

Backup

Tasks 20: Backup and Verify a Docker Volume

Volumes hold your persistent data — databases, uploaded files, certificates. Backing them up is not optional in a production environment. The tar method shown here is reliable, portable, and requires no extra tools beyond a standard Linux system.

The backup command uses a temporary container to compress the volume into a tarball on the host:

docker run --rm \
  -v part3_volume:/data:ro \
  -v $(pwd):/backup \
  ubuntu tar czf /backup/part3_volume-backup20260610.tar.gz -C /data .

What each flag does:

• • -rm: the helper container is automatically removed after the backup is done — no orphaned containers
• -v part3_volume:/data:ro: mounts the volume as read-only (important — no accidental modifications during backup)
• -v $(pwd):/backup: maps your current directory to the container’s /backup so the tarball lands on the host
• tar czf: create ©, gzip-compress (z), to a file (f) — standard compressed archive

Verify the backup by listing its contents — confirm all expected files are present:

tar -tzf part3_volume-backup20260610.tar.gz | head -n 20

💡 Best Practice: Automate volume backups with a cron job or a scheduled Swarm service. Store the backup tarballs off-host — in S3, Azure Blob Storage, or any remote storage. Test your restores periodically. A backup you’ve never restored from is an assumption, not a safety net.

💡 Use Case: Before any major update or destructive migration, run a manual volume backup. If something goes wrong, restoring is as simple as: docker run — rm -v part3_volume:/data -v $(pwd):/backup ubuntu tar xzf /backup/part3_volume-backup.tar.gz -C /data

Epilogue

I realize that this short 3-part intro to Docker may leave you a bit wanting for more. We have not made it to the main course yet! I am going to put something together on the Docker — AWS synergies, ECS, ECR, EKS, Fargate… Not sure if this will tie in to this series or if it will be standalone, as of yet. Let me know if there is anything specific you want me to dive into!

Previous: https://medium.com/@gkinthaert/a-day-as-a-docker-devops-engineer-117368c9f1eb

https://medium.com/@gkinthaert/a-day-as-a-docker-devops-engineer-lunch-ee1ea02eacdb