Docker From First Principles

How isolated Linux processes became portable software

In the **previous article**, we broke down one of the biggest misconceptions in modern software infrastructure:

A container is not a virtual machine, is not an Image and is not a docker.

**Read this article here for free**

At its core, a container is simply:

• a Linux process
• isolated using namespaces
• constrained using cgroups
• running on a shared kernel

But that immediately raises a bigger question:

If Linux already had containers internally…

Why did Docker completely change the software industry?

• Why did containers suddenly explode after Docker appeared?
• Why did companies rapidly rebuild infrastructure around it?
• And why did “Dockerizing applications” become standard engineering practice?

To answer that, we need to understand something important:

• Docker was never just about containers.
• Docker was about making environments portable.

Docker

The Real Problem Was Never Just Isolation

Container isolation alone was useful. But isolation was only half the problem. The bigger challenge was consistency.

Imagine two developers working on the same application.

Developer A has:

• Python 3.10
• Ubuntu 22
• specific system libraries

Developer B has:

• Python 3.8
• macOS
• slightly different dependencies

Now move the application into:

• staging
• production
• cloud infrastructure

Every environment becomes slightly different. This creates invisible instability.

The application itself may be correct… …but the runtime environment becomes unpredictable.

The industry desperately needed a way to package:

• application code
• dependencies
• runtimes
• system tools
• filesystem state

…into one reproducible unit. This is where Docker changed everything.

Docker’s Core Idea

Docker standardized containers into portable software artifacts.

Instead of saying:

“Install these dependencies and hopefully everything works…”

Developers could now say:

“Run this exact container image.”

This was a profound shift.

Infrastructure stopped being:

• manually assembled
• environment-specific
• operationally fragile

And started becoming:

• reproducible
• portable
• immutable
• automatable

This changed software delivery forever.

Docker Images: The Blueprint of a Container

One of Docker’s most important inventions was the container image.

A Docker image is NOT a running container. This distinction is critical.

A Docker image is:

a packaged filesystem snapshot + startup instructions

A running container is:

a live isolated process created from that image

Think of it like this:

Concept → Analogy
-----------------
Docker Image → Blueprint
Container    → Actual running building
Dockerfile   → Recipe to create blueprint

This mental model removes enormous confusion.

What Actually Exists Inside a Docker Image?

A Docker image can contain:

• application code
• binaries
• runtimes
• system libraries
• environment variables
• startup commands

For example:

Python App Image
├── Ubuntu filesystem
├── Python runtime
├── Flask
├── Application code
└── Startup command

When Docker starts a container:

• it creates isolated namespaces
• applies cgroup (control groups) limits
• mounts the image filesystem
• starts the application process

At the end of the day:

The container is still just a Linux process.

Docker simply automated and standardized the entire experience.

The Genius of Layered Images

This is one of Docker’s most elegant ideas. Docker images are built in layers.

For example:

Layer 1 → Ubuntu Base
Layer 2 → Python Runtime
Layer 3 → Flask Installation
Layer 4 → Application Code

Each layer is immutable. This creates massive efficiency benefits.

If only your application code changes:

• Docker reuses earlier layers
• only rebuilds the changed layer

This dramatically speeds up:

• builds
• deployments
• transfers
• caching

Layering became foundational to modern CI/CD systems.

Copy-on-Write: Why Containers Are Lightweight

When a container starts, Docker does NOT duplicate the entire image. Instead, the image layers remain read-only.

Docker adds:

a thin writable layer on top.

Like this:

Read-Only Image Layers
├── Ubuntu
├── Python
├── Flask
└── App Code

Writable Container Layer
└── Runtime Changes

This is called:

Copy-on-Write

Multiple containers can share the same underlying image layers efficiently. This is one reason containers are far lighter than virtual machines.

Dockerfiles: Infrastructure as Code

Docker introduced another transformative idea:

Infrastructure definitions became code. A Dockerfile describes how to build an image.

Example:

FROM python:3.11

WORKDIR /app

COPY . .

RUN pip install -r requirements.txt

CMD ["python", "app.py"]

This became incredibly powerful because environments became:

• version controlled
• reproducible
• reviewable
• automatable

Infrastructure stopped living only inside:

• shell scripts
• wiki pages
• tribal knowledge

Now environments themselves became portable artifacts.

Docker Daemon: The Hidden Engine

When you run:

docker run myapp

Docker itself is not the container.

Behind the scenes:

• Docker CLI talks to Docker daemon
• Docker daemon manages images
• Docker daemon interacts with container runtimes
• container runtime talks to Linux kernel

The stack looks roughly like this:

Docker CLI
    ↓
Docker Daemon
    ↓
Container Runtime
    ↓
Linux Kernel
    ↓
Namespaces + cgroups

This layering is important.

Because eventually the industry realized:

Docker itself didn’t need to manage everything.

This led to modern runtimes like:

• containerd
• CRI-O (Container Runtime Interface)

And later:

• Kubernetes integration

Registries: The Missing Piece

Portable images become powerful only when they can be distributed easily. This led to container registries. A registry stores container images.

Examples include:

• Docker Hub
• GitHub Container Registry
• Amazon Elastic Container Registry (ECR)
• Google Artifact Registry

Now software delivery became:

Developer builds image
        ↓
Push image to registry
        ↓
Servers pull identical image
        ↓
Run identical containers everywhere

This was revolutionary for distributed systems.

Why Developers Fell in Love with Docker

Docker dramatically simplified local development.

Before Docker:

• onboarding new engineers could take days
• dependency conflicts were constant
• local environments drifted over time

With Docker:

docker compose up

And suddenly:

• databases
• APIs
• queues
• caches

…all start consistently.

Entire development environments became reproducible. This massively accelerated engineering velocity.

The Hidden Shift Docker Created

Docker didn’t just introduce containers. It changed how engineers thought about infrastructure.

Applications became:

• ephemeral
• immutable
• portable

Servers became less important. The image became the deployment unit.

This shift directly enabled:

• microservices
• cloud-native systems
• modern CI/CD
• elastic scaling
• infrastructure automation

Docker normalized the idea that:

applications should move freely between environments.

That idea became foundational to modern cloud computing.

But Docker Introduced a New Problem

Docker solved:

• packaging
• portability
• reproducibility

But now companies faced another challenge.

What happens when you have:

• thousands of containers
• across hundreds of machines
• constantly crashing
• scaling
• updating
• communicating

How do you:

• restart failed containers?
• distribute workloads?
• balance traffic?
• manage networking?
• perform rolling deployments?

Managing containers manually quickly became impossible.

The industry now needed: **orchestration**.

And that is the problem Kubernetes was designed to solve. That’s where the next article begins.

The Mental Model to Keep

If the first article taught:

A container is an isolated Linux process

Then this article should leave you with a second foundational idea:

Docker transformed containers into portable, reproducible software artifacts.

That single idea explains why Docker changed the industry so dramatically. Containers provided isolation. Docker provided usability. And together, they became the foundation of modern infrastructure.

Read previous article about containers here

If you’re building data platforms, exploring analytics, or just love thinking about how data actually tells a story, feel free to follow or leave a clap 👏. It’s a small signal, but it helps me keep writing honest, example-driven content about data modelling, fact tables, dimensions, and the patterns that make analytics work.

Thanks for reading and for keeping curiosity alive ❤️.

Loved this article? I’ve created more in-depth resources to help you grow as a Data Engineer:

• **Data Engineering Roadmap — Guru, Eerla | Data Engineering**
• **Data Engineering Skills Guide — Guru, Eerla | Data Engineering**

Visit my site for the full collection.