What is Docker

Containers vs Virtual Machines

A virtual machine (VM) emulates an entire computer. Each VM runs a full guest operating system on top of a hypervisor (VMware ESXi, KVM, Hyper-V), consuming significant CPU, memory, and disk resources. Booting a VM takes minutes, and each VM requires its own OS kernel, libraries, and application binaries.

A container packages only the application and its dependencies into an isolated process. Containers share the host operating system's kernel through Linux namespaces and cgroups, making them far more lightweight. A container starts in milliseconds, uses a fraction of the memory, and can run hundreds of instances on hardware that would support only a handful of VMs.

Key Differences

Linux Primitives Behind Containers

Containers are not a Docker invention. They rely on two Linux kernel features:

• Namespaces provide process isolation. Each container gets its own PID, network, mount, UTS, IPC, and user namespace, so processes inside cannot see or affect the host or other containers.
• cgroups (control groups) limit and account for resource usage. They prevent a single container from consuming all CPU, memory, or I/O on the host.

Together, namespaces and cgroups create the illusion of a lightweight VM while actually being isolated user-space processes.

When to Use Containers Over VMs

Containers excel for microservices, CI/CD pipelines, consistent dev/prod environments, and scaling stateless applications. VMs remain appropriate when you need different operating system kernels, strong security boundaries, or hardware-level isolation for compliance reasons.

Docker Architecture

Docker uses a client-server architecture with three major components:

Docker Daemon (dockerd)

The Docker daemon (dockerd) runs as a background service on the host machine. It listens for Docker API requests and manages Docker objects: images, containers, networks, and volumes. The daemon can also communicate with other daemons to manage distributed Docker services (Swarm mode).

$ systemctl status docker
docker.service - Docker Application Container Engine
   Active: active (running) since Mon 2024-01-15 10:00:00 UTC
   Main PID: 1234 (dockerd)

Docker Client (docker)

The Docker client (docker) is the primary way users interact with Docker. When you run docker run, docker build, or docker pull, the client sends these commands as REST API calls to the Docker daemon. The client can communicate with a local or remote daemon.

$ docker --version
Docker version 24.0.7, build afdd53b

$ docker info
Client: Docker Engine - Community
 Context:    default
 Server Version: 24.0.7
 Storage Driver: overlay2

Docker Registry

A Docker registry stores Docker images. Docker Hub is the default public registry, but organizations run private registries (Docker Registry, Harbor, AWS ECR, GCR, ACR). When you run docker pull nginx, the client contacts the registry, downloads the image layers, and stores them locally.

$ docker pull nginx:1.25
1.25: Pulling from library/nginx
a2abf6c4d29d: Pull complete
a9edb18cadd1: Pull complete
589b7251471a: Pull complete
Digest: sha256:4c0bedf09164d35...
Status: Downloaded newer image for nginx:1.25

The Docker Workflow

1. You write a Dockerfile defining your image build steps
2. docker build sends the build context to the daemon, which executes each instruction as a new layer
3. The resulting image is stored locally
4. docker run creates a container from that image with an isolated filesystem and network
5. The container runs the CMD or ENTRYPOINT process

This separation of client, daemon, and registry allows teams to share images, automate builds, and deploy consistently across environments.

Images, Containers, and Layers

What is a Docker Image?

A Docker image is a read-only template containing an application, its runtime, libraries, environment variables, and filesystem. Images are built in layers — each instruction in a Dockerfile creates a new layer stacked on top of the previous ones.

# Layer 1: Base image
FROM node:20-alpine
# Layer 2: Working directory
WORKDIR /app
# Layer 3: Copy dependency files
COPY package*.json ./
# Layer 4: Install dependencies
RUN npm ci --only=production
# Layer 5: Copy application code
COPY . .
# Layer 6: Set command
CMD ["node", "server.js"]

Each layer is cached independently. If you change only your application code (layer 5), Docker reuses layers 1-4 from cache, making rebuilds extremely fast.

What is a Docker Container?

A container is a runnable instance of an image. When you run docker run, Docker creates a container by:

1. Pulling the image if not present locally
2. Creating a writable container layer on top of the image layers
3. Allocating a filesystem, network interface, and process space
4. Starting the container's main process

$ docker run -d --name web -p 8080:80 nginx:1.25
a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6q7r8s9t0

The container's writable layer captures any changes made during runtime (new files, modified config, logs). This layer is ephemeral — destroying the container removes all changes unless you committed them to a new image or mounted a volume.

Copy-on-Write Strategy

Docker uses a copy-on-write (CoW) strategy. When a running container needs to modify a file from an image layer, Docker copies the file to the writable layer first, then modifies the copy. The original image layer remains unchanged. This keeps images immutable and shareable.

Image Naming Convention

Images follow the format: [registry/]repository[:tag|@digest]

nginx                    # Docker Hub official image, :latest tag
myregistry.com/myapp:v2  # Private registry, custom tag
nginx@sha256:abc123...    # Specific image digest

The :latest tag is the default but should never be used in production — it creates ambiguity about which version is actually running.

The Container Ecosystem

OCI Standards

The Open Container Initiative (OCI) defines open industry standards for container formats and runtimes:

• Runtime Specification (runtime-spec): Defines how to run a container — configuration, lifecycle, and execution environment
• Image Specification (image-spec): Defines the image format — manifest, layers, and configuration
• Distribution Specification (distribution-spec): Defines the API for pushing and pulling images from registries

Docker was instrumental in creating OCI. Today, alternatives like Podman, CRI-O, and containerd all implement OCI standards, meaning images built with Docker can run on any OCI-compliant runtime.

containerd and runc

Docker Engine internally uses two key components:

• containerd: A container runtime that manages the complete container lifecycle — image transfer, container execution, supervision, and networking. Docker Engine delegates actual container work to containerd.
• runc: A lightweight OCI-compliant runtime that actually creates and runs containers. containerd spawns runc to start each container process.

$ ctr containers list
CONTAINER    IMAGE          STATUS
a1b2c3d4     nginx:1.25     RUNNING

This modular architecture means containerd can be used independently of Docker for Kubernetes workloads.

Docker Desktop vs Docker Engine

• Docker Desktop (Mac, Windows): A GUI application that includes Docker Engine, Docker CLI, Docker Compose, Kubernetes, and a VM to run Linux containers. On macOS and Windows, Docker Desktop runs a Linux VM because containers are Linux-native.
• Docker Engine (Linux): The native Docker daemon running directly on the Linux kernel. Preferred for servers and production deployments.

Comparison with Alternatives

Understanding this ecosystem helps you choose the right tool. For most development workflows, Docker remains the standard. For Kubernetes production, containerd or CRI-O handles runtime duties.

Running Your First Container

Hello World

The simplest Docker command downloads an image and runs it:

$ docker run hello-world

Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
c1ec31eb5944: Pull complete
Digest: sha256:...
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

What happened behind the scenes:

1. Docker client contacted the daemon
2. Daemon checked for hello-world:latest locally — not found
3. Daemon pulled the image from Docker Hub
4. Daemon created a container from the image
5. Container ran its CMD, printed the message, and exited

Interactive Container

To explore a container interactively:

$ docker run -it alpine sh
/ # uname -a
Linux a1b2c3d4 6.5.0 #1 SMP x86_64 Linux
/ # cat /etc/os-release
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.19.0
/ # exit

The -i flag keeps STDIN open, and -t allocates a pseudo-TTY. Together they give you an interactive shell.

Practical Examples

Run a web server and access it:

$ docker run -d --name nginx-web -p 8080:80 nginx:1.25
# Visit http://localhost:8080 in your browser

Run a database:

$ docker run -d --name postgres-db \
  -e POSTGRES_PASSWORD=secret \
  -e POSTGRES_DB=myapp \
  -p 5432:5432 \
  -v pgdata:/var/lib/postgresql/data \
  postgres:16

Run a one-off command:

$ docker run --rm python:3.12-slim python -c "print('Hello from Python')"
Hello from Python

The --rm flag automatically removes the container when it exits, preventing accumulation of stopped containers.

Container Lifecycle

A container progresses through states: created → running → paused → stopped → removed. Understanding these states helps you manage containers effectively:

$ docker ps -a
CONTAINER ID   IMAGE          STATUS                     NAMES
a1b2c3d4e5f6   nginx:1.25     Up 5 minutes               nginx-web
b2c3d4e5f6g7   alpine         Exited (0) 2 minutes ago   romancing_alpini
c3d4e5f6g7h8   postgres:16    Up 10 minutes              postgres-db