From Containers to Docker-in-Docker: Understanding the Why and When

Photo by Rubaitul Azad on Unsplash

Problem: Containers Were Hidden Gems

Before kubernetes, many companies knew about containers, but they were the secret sauce running behind the scenes at big tech companies, offering consistent environments and fast deployment speeds.

That all changed with Docker.

Docker didn’t invent containers, but it made them simple, portable, and developer-friendly. What once required deep system knowledge and complex tooling was now as easy as writing a Dockerfile and running docker build.

Docker democratized containers and, more importantly, helped bridge the gap between Devs and Ops.

What is docker?

In simple terms, docker takes an application and everything it needs (dependencies and runtime), then puts them in an isolated environment. This makes the application independent of other processes and enables it to run smoothly across different environments (servers).

A Real-World Example:

Let’s say you’re developing a Python web app.

Without Docker:

• You install Python locally.
• You manually install dependencies with pip install.
• You might hit version conflicts with other apps.

With Docker:

• You write a Dockerfile that installs Python, sets up dependencies, and starts the app.
• You build an image and run a container.
• No matter where you run it — on your laptop, staging, or the cloud — it behaves the same.

What are the benefits?

1. dependency management. You can now build an image from a Dockerfile that defines everything you need rather than simply installing everything. The code will run no matter where it is running.
2. Isolation. Apps running in different containers can have different mini-OS even if they rely on conflicting libraries or runtimes.
3. Portability. Your app runs the same in dev, test, or prod.
4. Fast. Containers start up in seconds.

What is DinD?

Normally, we run docker commands on host system. But for DinD(Docker in Docker), we run docker commands inside a container. The need behind this over-nested virtualization is mainly for CI. So your CI job spins up a container that includes Docker itself, so you can build and run Docker images from inside your pipeline.

Note: you will usually need a privileged container to run docker.

When should you use it and when shouldn’t?

Use DinD When:

• You need to build Docker images inside a CI job.
• Your CI environment doesn’t allow privileged access to the host system.
• You’re okay with performance trade-offs and potential security concerns.

Avoid DinD When:

• You’re not using CI — for example, running local development or production containers.

If you found this helpful, feel free to give it a clap (or two!) and follow me TechWithPlum on Medium for more practical dev content on Docker, Kubernetes, CI/CD, and cloud-native tools. 🚀