Docker: The Complete Guide

What is Docker?

Docker is a platform for building, shipping, and running applications in containers. It's not just a tool—it's an entire ecosystem that revolutionized how we deploy software.

                    The Docker Promise: "Build once, run anywhere." Package your application with all
                    its dependencies into a standardized unit that works identically on any machine.
                

Released in 2013, Docker made Linux containers accessible to developers. Before Docker, using containers required deep kernel knowledge and complex scripts. Docker wrapped everything in a simple API.

Images vs Containers: The Simple Truth

An Image is a template. A Container is a running instance of that template.

                    Think of Virtual Machines:
                    Image = VM Template (sitting on disk, not running)
Container = Running VM (active, using resources)

                    From ONE template, you can create MULTIPLE running VMs. Same with Docker: one image → many
                    containers.
                

Core Concepts

1. Images: The Blueprint

A Docker Image is a read-only template containing everything needed to run an application: code, runtime, libraries, environment variables, and configuration files.

                    
# List all images on your system

$ docker images

REPOSITORY    TAG       IMAGE ID       CREATED        SIZE

nginx         latest    abc123def456   2 weeks ago    142MB

python        3.11      def456ghi789   1 month ago    917MB

ubuntu        20.04     ghi789jkl012   2 months ago   72.8MB

Images are built in layers. Each instruction in a Dockerfile creates a new layer. Layers are cached and reused, making builds fast.

💡 Pro Tip: Put frequently changing files (like source code) in LATER layers. Put rarely changing files (like dependencies) in EARLIER layers. This maximizes cache hits!

Copy-on-Write: Docker's Storage Secret

Copy-on-Write (COW): A technique where multiple containers can share the same image layers in read-only mode. When a container modifies a file, Docker copies only that file to the container's writable layer.

This is how Docker achieves incredible efficiency. When you run 10 containers from the same image, you don't get 10 copies of the image—you get 10 containers sharing the same read-only layers.

How It Works

Every container has two types of layers:

Read-only layers: Shared image layers (from the Dockerfile)
Writable layer: Container-specific layer for changes

The COW Process: What Happens When You Modify a File

                    
# Inside Container 1

$ echo "Hello" > /app/config.txt  # File exists in Layer 3

# Docker's COW mechanism:

# 1. Finds config.txt in read-only Layer 3

# 2. Copies ONLY config.txt to Container 1's writable layer

# 3. Modifies the copy in the writable layer

# 4. Original file in Layer 3 remains unchanged

# 5. Container 2 still sees the original file

Why This Matters

Real-World Impact:

Storage Efficiency: Running 100 containers from a 500MB image uses ~500MB + (100 × writable layer size), not 50GB
Fast Startup: Creating a container is instant—no copying needed
Build Speed: Unchanged layers are reused across builds

COW in Action: A Practical Example

                    
# Build an image

$ docker build -t myapp .

# Run 3 containers from the same image

$ docker run -d --name app1 myapp

$ docker run -d --name app2 myapp

$ docker run -d --name app3 myapp

# Check disk usage

$ docker system df

TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE

Images          1         1         385MB     0B (0%)

Containers      3         3         15MB      0B (0%)

# Only 385MB (image) + 15MB (3 writable layers) = 400MB total!

# Without COW: 385MB × 3 = 1.1GB

The Downside: Write Performance

⚠️ Performance Consideration: The first time a container modifies a file, Docker must copy it from the image layer to the writable layer. This "copy-up" operation can be slow for large files.

Solution: For write-heavy workloads (databases, logs), use volumes instead of the container's writable layer. Volumes bypass COW entirely.

                    
# Bad: Database writes to container layer (slow COW)

$ docker run -d postgres

# Good: Database writes to volume (no COW)

$ docker run -d -v pgdata:/var/lib/postgresql/data postgres

Viewing Layer Information

                    
# Inspect image layers

$ docker history myapp

IMAGE          CREATED        SIZE      COMMENT

abc123def456   2 hours ago    15MB      COPY . .

def456ghi789   2 hours ago    250MB     RUN pip install

ghi789jkl012   2 weeks ago    120MB     FROM python:3.11

# See container's writable layer size

$ docker ps -s

CONTAINER ID   IMAGE    SIZE

abc123         myapp    5MB (virtual 390MB)

#                    ↑ writable layer

#                           ↑ total (image + writable)

2. Containers: The Running Instance

A Container is a running instance of an image. It's like the relationship between a class and an object in programming.

                    
# Run a container from an image

$ docker run -d --name myapp nginx

# List running containers

$ docker ps

CONTAINER ID   IMAGE     COMMAND                  STATUS         PORTS

abc123def456   nginx     "nginx -g 'daemon of…"   Up 5 minutes   80/tcp

3. Dockerfile: The Recipe

A Dockerfile is a text file containing instructions to build an image. It's your application's recipe.

                    
# Dockerfile

FROM python:3.11-slim

WORKDIR /app

COPY requirements.txt .

RUN pip install --no-cache-dir -r requirements.txt

COPY . .

EXPOSE 8000

CMD ["python", "app.py"]

4. Registry: The Distribution Hub

A Registry is where images are stored and shared. Docker Hub is the public registry, but you can run private ones too.

                    
# Pull an image from Docker Hub

$ docker pull nginx:latest

# Push your image to a registry

$ docker tag myapp:latest username/myapp:v1.0

$ docker push username/myapp:v1.0

The Docker Workflow

Understanding the typical Docker development workflow:

📝

Write
Dockerfile

→

🔨

Build
docker build

→

📤

Push
docker push

→

🚀

Run
docker run

                    
# 1. Build an image from Dockerfile

$ docker build -t myapp:v1.0 .

# 2. Test it locally

$ docker run -p 8000:8000 myapp:v1.0

# 3. Tag for registry

$ docker tag myapp:v1.0 myregistry.com/myapp:v1.0

# 4. Push to registry

$ docker push myregistry.com/myapp:v1.0

# 5. Deploy on production server

$ docker pull myregistry.com/myapp:v1.0

$ docker run -d -p 80:8000 myregistry.com/myapp:v1.0

Essential Docker Commands

The commands you'll use every day:

Build an Image

docker build -t name:tag .

Run a Container

docker run -d -p 8080:80 nginx

List Containers

docker ps -a

Stop a Container

docker stop <container-id>

Remove a Container

docker rm <container-id>

View Logs

docker logs -f <container-id>

Execute Command

docker exec -it <id> bash

Inspect Container

docker inspect <container-id>

Common Flags Explained

-d : Run in detached mode (background)
-p 8080:80 : Map host port 8080 to container port 80
-v /host:/container : Mount a volume
--name myapp : Give the container a name
-e KEY=VALUE : Set environment variable
--rm : Automatically remove container when it stops
-it : Interactive terminal (for shells)

Understanding Dockerfile: The WHY Behind Best Practices

As a beginner, you might wonder: "Why do these rules matter?" Let's understand the reasoning behind each practice.

1. Why Use Specific Base Images?

The Problem: If you use FROM python, Docker will pull whatever version is tagged as "latest" today. In 6 months, "latest" might be Python 4.0, and your app breaks.

What happens: Your teammate builds the image next week and gets a different Python version. Your code works on your machine but fails on theirs.

The Solution: Pin the exact version: FROM python:3.11-slim

3.11 = Exact Python version
slim = Minimal variant (smaller size, fewer vulnerabilities)

❌ Unpredictable

FROM python

Could be 3.9, 3.11, or 4.0

✅ Reproducible

FROM python:3.11-slim

Always Python 3.11

2. Why Minimize Layers?

How Docker Works: Each RUN command creates a new layer. Each layer is stored separately. More layers = bigger image = slower downloads.

The Problem with Multiple RUN Commands:

                    
RUN apt-get update          # Layer 1: 30MB

RUN apt-get install curl    # Layer 2: 10MB

RUN rm -rf /var/lib/apt     # Layer 3: 0MB (but Layer 1 & 2 still exist!)

# Total: 40MB (cleanup didn't help!)

Why cleanup doesn't work: Docker layers are immutable. When you delete files in Layer 3, Layers 1 and 2 still contain the original files. They're hidden, but they still take up space in the image.

The Solution: Chain commands with && to create a single layer:

                    
RUN apt-get update && \

    apt-get install -y curl && \

    rm -rf /var/lib/apt/lists/*

# Total: 10MB (cleanup works because it's in the same layer!)

3. Why Order Matters: Build Cache

Docker's Cache System: Docker caches each layer. If a layer hasn't changed, Docker reuses the cached version instead of rebuilding it.

The Problem: You change one line of code. Docker rebuilds EVERYTHING.

                    
# Bad Order

FROM node:18

COPY . .                    # Copies ALL your code

RUN npm install             # Installs dependencies

# You change app.js → Cache breaks → npm install runs again (slow!)

How to Fix It: Copy dependencies first, then code.

                    
# Good Order

FROM node:18

# Step 1: Copy only package files

COPY package*.json ./

RUN npm install             # Cached unless package.json changes

# Step 2: Copy code (changes frequently)

COPY . .

# You change app.js → Only Step 2 rebuilds (fast!)

4. What is .dockerignore and Why Use It?

The Problem: COPY . . copies EVERYTHING in your project folder into the image.

node_modules/ (500MB of dependencies you'll reinstall anyway)
.git/ (your entire Git history)
*.log (debug logs)
.env (secrets that shouldn't be in the image!)

The Solution: Create a .dockerignore file (like .gitignore):

                    
# .dockerignore

node_modules

.git

*.log

.env

README.md

.vscode

Now COPY . . skips these files. Your image is smaller and faster to build.

5. Multi-Stage Builds: The Secret to Tiny Images

The Problem: To build a Go app, you need the Go compiler (800MB). But to RUN the app, you only need the compiled binary (10MB).

Without Multi-Stage: Your final image contains the compiler, source code, and everything else. Total: 850MB.

With Multi-Stage: Build in one "stage", copy only the binary to a fresh "stage". Total: 15MB.

                    
# Stage 1: Build Environment (has compiler)

FROM golang:1.20 AS builder

WORKDIR /app

COPY . .

RUN go build -o myapp       # Creates a 10MB binary

# Stage 2: Production (starts FRESH, no compiler!)

FROM alpine:3.18

COPY --from=builder /app/myapp /usr/local/bin/

CMD ["myapp"]

# Final image: Alpine (5MB) + binary (10MB) = 15MB

# The 800MB compiler is left behind in Stage 1!

How it works: Docker builds both stages, but only the LAST stage becomes your final image. The first stage is discarded (but you can copy files from it using --from=builder).

Docker Compose: Multi-Container Apps

Most applications need multiple services (web server, database, cache). Docker Compose lets you define and run multi-container applications.

                    
# docker-compose.yml

version: '3.8'

services:

  web:

    build: .

    ports:

      - "8000:8000"

    environment:

      - DATABASE_URL=postgres://db:5432/myapp

    depends_on:

      - db

      - redis

  db:

    image: postgres:15

    environment:

      POSTGRES_PASSWORD: secret

    volumes:

      - db-data:/var/lib/postgresql/data

  redis:

    image: redis:7-alpine

volumes:

  db-data:

                    
# Start all services

$ docker-compose up -d

# View logs

$ docker-compose logs -f web

# Stop all services

$ docker-compose down

# Rebuild and restart

$ docker-compose up --build

Volumes: Persistent Data

Containers are ephemeral. When you delete a container, its data is gone. Volumes solve this by storing data outside the container.

Types of Mounts

Named Volumes: Managed by Docker. Best for production.
docker run -v mydata:/app/data nginx
Bind Mounts: Map a host directory. Good for development.
docker run -v /host/path:/container/path nginx
tmpfs Mounts: Stored in memory. Fast but temporary.
docker run --tmpfs /app/cache nginx

                    
# Create a volume

$ docker volume create mydata

# Use it in a container

$ docker run -d -v mydata:/data postgres

# List volumes

$ docker volume ls

# Inspect a volume

$ docker volume inspect mydata

# Remove unused volumes

$ docker volume prune

Networking: Container Communication

Docker creates virtual networks to allow containers to communicate.

Network Types

bridge (default): Containers on the same bridge can talk to each other.
host: Container uses the host's network directly. No isolation.
none: No network access at all.

                    
# Create a custom network

$ docker network create mynetwork

# Run containers on the same network

$ docker run -d --name db --network mynetwork postgres

$ docker run -d --name web --network mynetwork nginx

# Now 'web' can reach 'db' by hostname!

$ docker exec web ping db  # Works!

Summary

Docker transformed software deployment by making containers accessible. Key takeaways:

Images are blueprints, Containers are running instances
Dockerfiles define how to build images
Compose orchestrates multi-container apps
Volumes persist data, Networks connect containers
Always optimize for size, security, and cache efficiency