A day as a Docker DevOps Engineer: lunch

Part II of III: Storage, Networking, and Docker Compose

Introduction

Containers are ephemeral by design — when they stop, everything inside them disappears. That’s great for stateless services, but the moment your app needs to store data that outlives a restart, you need volumes.

Networking is the other side of the same coin: isolated containers need a way to talk to each other without exposing ports to the outside world.

In Part 2 we cover Docker volumes, custom networks, and Docker Compose — the trio that turns a collection of independent containers into a working application stack. This is all about keeping data alive, wiring containers together, and orchestrating multi-service deployments

Storage

Task 6: Create a Docker Volume

A Docker volume is persistent storage that lives outside the container lifecycle. Even if you stop, remove, and recreate a container, the data on a volume survives.

docker volume create some_volume   (create a volume)
docker volume ls  (lists the volumes)

Inspect the volume to see where Docker is actually storing the data on the host:

docker volume inspect some_volume

You want to see the volume capabilities with your own eyes at least once, so we will mount the volume to an ubuntu container’s /data directory and create some test files:

docker run -dit -v some_volume:/data ubuntu bash
docker ps (you should see your container running)
docker exex -it [container name] bash (access the container)

ls to the /data directory and go ahead and create some test files of your choice

• dit: The -d flag causes Docker to start the container in “detached” mode. A simple way to think of this is to think of -d as running the container in “the background,” just like any other Unix process. To run a container in interactive mode, you use the docker run -it command. Here’s what each flag stands for: -i flag: Keeps the standard input (stdin) open, allowing you to send commands to the container. -t flag: Allocates a pseudo-terminal (tty), providing a terminal interface for executing commands interactively.
• bash: we want to get into the container to create some files…

Now that you have created some files inside the container, exit that container, and go ahead and stop and remove it (you need to stop a container first before you can remove):

docker stop [container id/container name] && docker rm [container id/container name]

docker ps (to verify that the container is stopped), or skip and 
docker ps -a (to verify that the container is no longer available in any state)

The volume was not deleted, just the container. We can verify that the data (the files) we created, are still present on the volume!

sudo ls -la /var/lib/docker/volumes/some_volume/_data

💡 Why This Matters: Volumes are how databases, uploaded files, application logs, and any other stateful data survive container restarts and updates. Without volumes, a simple docker stop on a database container would wipe all your data.

Task 7: Demonstrate Data Persistence

This is the key test. Spin up a brand-new container — different name, same volume — and confirm the files from the previous container are still there, by navigating to /data inside the new container:

docker run -dit -v some_volume:/data --name new-device ubuntu bash
docker exec -it [container name] bash

💡 Best Practice: Use named volumes (part3_volume) rather than bind mounts (/host/path:/container/path) for application data in Docker-managed environments. Named volumes are portable, easier to back up, and work the same on any host regardless of the host directory structure.

Networking

Task 8: Create a Custom Docker Network

By default, containers on the default bridge network can communicate by IP but not by name. A custom bridge network gives you DNS-based name resolution, better isolation, and cleaner architecture.

docker network create --driver bridge newcustomnetwork
docker network ls

💡 Use Case: Custom networks are essential in multi-tier applications. A typical setup puts your web frontend and API backend on a frontend network, and your API and database on a separate backend network. The database is never reachable from the frontend directly — the network topology enforces the security boundary.

Task 9: Inter-Container Communication by Name

This is one of Docker networking’s most useful features: containers on the same custom network can reach each other using their name as a hostname. No hardcoded IPs, no manual /etc/hosts editing.

Let’s deploy two ubuntu containers on the custom network we just created (I am using tail -f /dev/null to keep these running btw, as containers without an ongoing task will do their thing and then assume ‘exited’ state:

docker run -d --name customapp1 --network newcustomnetwork ubuntu tail -f /dev/null
docker run -d --name customapp2 --network necustomnetwork ubuntu tail -f /dev/null

Verify that both of these containers are running on the specified network, by taking a look at the ‘Containers’ section of the JSON output :

docker inspect newcustomnetwork

Now you can showcase this feature by installing ping on each container and pinging the other container and vice-versa, by name!

docker exec -it [container name] bash (access the container)
apt update && apt install -y iputils-ping (install ping)

💡 Best Practice: Always use container names (not IPs) when wiring up inter-service communication. Container IPs can change when containers are recreated. Container names on a custom network provide stable, predictable DNS entries.

Docker Compose

Task 10: Create and Deploy a Compose Stack

Docker Compose is where the individual pieces — containers, volumes, networks — come together into a single declarative file. Instead of running five docker run commands in the right order, you define your entire stack in compose.yaml (or compose.yml) and start it with one command. (btw you can use another name for this file, but you have to declare that name:

docker compose -f my-custom-name.yaml up

Why bother? This is helpful if you want to keep environments separate

docker compose -f compose_test.yaml -f compose_prod.yaml up

While we are here, let’s take a look at the important Docker compose commands:

Core Lifecycle Commands
docker compose up — Creates and starts all containers.
docker compose down — Stops and removes containers, networks, and volumes.
docker compose stop — Pauses running containers without deleting them.
docker compose start — Restarts stopped containers.
docker compose restart — Restarts running containers.

Monitoring and Debugging
docker compose ps — Lists running containers and their statuses.
docker compose logs — Displays log outputs from all services.
docker compose top — Displays running processes inside containers.
docker compose exec — Runs a command inside a specific running container.
docker compose run — Runs a one-off command inside a new container.

Building and Managing
docker compose build — Builds or rebuilds image files for your services.
docker compose pull — Downloads required images from a registry.
docker compose config — Validates and views the final compiled Compose file.

This is the content of the example compose.yaml file:

services:
  nginx:
    image: nginx:latest
    container_name: nginx_container
    ports:
      -  "8085:80"
    volumes:
      -  new-volume:/usr/share/nginx/html/data
    networks:
      -  new-bridge-network
    depends_on:
      -  mysql
  mysql:
    image: mysql:8
    container_name: mysql_container
    environment:
      MYSQL_ROOT_PASSWORD: 1234root
      MYSQL_DATABASE: mydb
    networks:
      - new-bridge-network    

networks:  
  new-bridge-network:
    driver: bridge

volumes:
  new-volume:

## this is a simple example, but in real life you would secure the DB passwords
## option 1.  Use .env with following sample content
### MYSQL_ROOT_PASSWORD=supersecretpassword
### MYSQL_DATABASE=mydb
## add the .env to .gitignore
## reference it in the compose.yaml file with variables
# mysql:
#  image: mysql:8
#  environment:
#    MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
#    MYSQL_DATABASE: ${MYSQL_DATABASE}

## option 2. create a secret:
### echo "my-secret-password" > db_password.txt
## add this section in the compose.yaml
#

#    image: mysql:8
#    secrets:
#      - db_password

#secrets:
#  db_password:
#    file: ./db_password.txt

• nginx:latest — the web frontend
• mysql:8 — the database backend
• new-bridge-network — a custom network for internal communication
• new-volume — a named volume for MySQL data persistence
• Port 8085 — the exposed host port

Now that you have the compose.yaml file ready, run the following command:

docker compose up -d  (-d for detached mode)

Now you should see the new volume, the network, the container, and you willbe able to access the nginx welcome page on your EC2 instance public IP and port 8085 as well! (just make sure your security group allows http)

docker ps (to see the new containers)
docker volume ls (to see the volumes)
docker network ls (to see the network)

See the 2 new containers!

See the volume!

See the new network!

💡 Best Practice: Use docker compose up -d for background startup and docker compose down (not docker stop) to cleanly shut down a stack. docker compose down also removes the network. Add — volumes to also remove named volumes if you want a full clean slate.

💡 Use Case: Compose is the standard tool for local development environments. Your teammates clone the repo, run docker compose up, and have the full application stack running in seconds — same versions, same config, every time. No more “works on my machine.”

Don’t forget to clean up your environment! This is definitely something you want to do as you quickly run out of space, and we have a lot more to take a look at, in the next blog…

Make sure no containers are running!
docker ps (see list of running containers)
docker stop [container id / container name]
docker rm [container id / container name]
docker ps -a will show stopped containers too, worth a look!
docker rmi [image name]  removes images

docker system prune — Removes all stopped containers, unused networks, and dangling images (images without a tag).
docker system prune -a — Removes everything above, plus any unused image files not currently associated with a running container.
docker system prune --volumes — Includes unused volumes in the cleanup (by default, system prune leaves volumes untouched to protect your data).
docker system prune -a --volumes — The ultimate cleanup: wipes absolutely everything not currently running.

Previous episode: https://medium.com/@gkinthaert/a-day-as-a-docker-devops-engineer-117368c9f1eb

Next up is afternoon tea: part III: Swarm, Secrets, Resource Management, Monitoring & Backup