Containers Under the Hood: Cgroups, Namespaces & Linux Kernel

What IS a Container? (The Big Picture)

Before we dive into Linux internals, let's understand what problem containers actually solve.

The Problem: "It Works on My Machine"

Imagine you build an application on your laptop. It works perfectly. You send it to your colleague, and it crashes. Why?

Different Python version (you have 3.9, they have 3.7)
Missing system libraries (you have libssl 1.1, they have 1.0)
Different environment variables
Different file paths

This is the dependency hell problem. Your code doesn't run in isolation—it depends on the environment.

The Solution: Package Everything Together

A container is a self-contained execution environment. It bundles:

📦 Your application code
📚 All runtime dependencies (Python, Node.js, etc.)
🔧 System libraries (libc, libssl, etc.)
⚙️ Configuration files
🌍 Environment variables

                The Promise: If it runs in a container on your laptop, it will run identically in a
                container on the server. The environment is frozen and portable.
            

Containers vs. VMs: The Key Difference

You might think: "Isn't that just a Virtual Machine?" Not quite.

Virtual Machine

Virtualizes hardware
Runs a full OS kernel
Heavy (GBs)
Slow boot (minutes)
Strong isolation

Container

Virtualizes OS
Shares the host kernel
Lightweight (MBs)
Instant boot (milliseconds)
Process-level isolation

The Three Pillars of Containerization

To create this illusion of isolation, any container technology (Linux, BSD, Windows) needs three capabilities:

1. Isolation (Namespaces)

Make the process think it's alone. It can't see other processes, has its own network, its own filesystem view.

2. Resource Limits (Cgroups)

Prevent one container from hogging all CPU/RAM. Set hard limits on what it can consume.

3. Efficient Storage (Layering)

Don't duplicate the entire OS for each container. Use Copy-on-Write to share common files.

Now that we understand what a container is and why it exists, let's see how Linux implements these three pillars.

Are Containers Just for Linux?

This is a crucial distinction: "Container" is a generic concept, but Docker is a specific implementation built on Linux.

The Universal Concept: Process Isolation

The idea of "sandboxing" a process so it can't mess with the rest of the system is not unique to Linux. It is a fundamental Operating System design pattern that has appeared in many forms:

FreeBSD Jails (2000): One of the earliest implementations. Allowed partitioning a FreeBSD system into independent mini-systems.
Solaris Zones (2004): A very advanced implementation of application isolation on Sun Solaris systems.
AIX Workload Partitions: IBM's version for their AIX operating system.

So why do we associate Containers with Linux?

Because Linux is the OS that powers the cloud. When Docker arrived in 2013, it didn't invent the concept. It just made the existing Linux Kernel features (Namespaces & Cgroups) easy to use for developers.

Before Docker, using these features required complex shell scripts and deep kernel knowledge. Docker wrapped them in a friendly API (docker run), and that's why "Container" became synonymous with "Linux Container" in the public mind.

                The Windows/Mac Reality:
                
                When you run Docker on macOS or Windows, you are actually running a hidden Linux Virtual
                    Machine. Your containers run inside that Linux VM because they rely on the specific
                Linux implementation of namespaces.
                
                Note: Windows now has "Windows Containers" which use Windows-specific kernel primitives to
                    achieve the same goal, proving again that the concept is universal, even if the
                    implementation differs.

1. Namespaces: The Walls

Namespaces are the feature that provides Isolation. They limit what a process can see. When you wrap a process in a namespace, it's like putting it in a room with no windows.

PID

MNT

NET

IPC

PID 1

Linux provides 7 types of namespaces. Let's explore each one in detail.

PID Namespace: Process Isolation

The PID namespace gives each container its own process tree. Inside the namespace, the first process becomes PID 1 (like the system's init process).

                
# On the Host

$ ps aux | grep nginx

root  24567  nginx: master process

# Inside the Container

$ ps aux

USER  PID   COMMAND

root  1     nginx: master process  # Same process, different PID!

Why PID 1 matters: In Unix, PID 1 is special. It's the "reaper" of orphaned processes. If a parent process dies, its children get adopted by PID 1. This is why containers often use init systems like tini or dumb-init.

MNT Namespace: Filesystem Isolation

Each container gets its own view of the filesystem. The container's root (/) is completely separate from the host's root.

                
# Create a new mount namespace and chroot

$ unshare --mount --uts --ipc --net --pid --fork bash

# Now you're in a new namespace

$ mount --bind /new/rootfs /new/rootfs

$ chroot /new/rootfs /bin/bash

# You now see a completely different filesystem!

NET Namespace: Network Isolation

Each container gets its own network stack: interfaces, IP addresses, routing tables, and firewall rules.

                
# Create a network namespace

$ ip netns add container1

# List network interfaces in the namespace (empty!)

$ ip netns exec container1 ip link

1: lo: <LOOPBACK> mtu 65536

# Create a veth pair to connect it

$ ip link add veth0 type veth peer name veth1

$ ip link set veth1 netns container1

$ ip netns exec container1 ip addr add 10.0.0.2/24 dev veth1

UTS Namespace: Hostname Isolation

Allows each container to have its own hostname and domain name.

                
# Host

$ hostname

my-server

# Container

$ docker run -it --hostname=webapp ubuntu bash

root@webapp:/# hostname

webapp  # Different hostname!

IPC Namespace: Inter-Process Communication

Isolates System V IPC objects (message queues, semaphores, shared memory).

USER Namespace: User ID Remapping

This is the security game-changer. It allows a process to be root (UID 0) inside the container but map to an unprivileged user (e.g., UID 100000) on the host.

                Security Win: Even if an attacker escapes the container as "root", they're actually
                just user 100000 on the host with no privileges!
            

Cgroup Namespace: Resource View Isolation

Hides the cgroup hierarchy from the container, making it think it's at the root of the cgroup tree.

2. Cgroups: The Budget

If Namespaces limit what you can see, Control Groups (Cgroups) limit what you can use.

Without Cgroups, a containerized process could still consume 100% of the CPU or eat up all the RAM, crashing the host. Cgroups act as a meter and a throttle.

CPU Limit: 0.5 Core

RAM Limit: 512MB

Cgroup Controllers

Cgroups are organized into controllers, each managing a different resource type:

cpu: Limits CPU time. Uses shares (relative weight) or quotas (hard limits).
memory: Limits RAM usage. Can trigger OOM killer if exceeded.
blkio: Limits disk I/O bandwidth and operations per second.
pids: Limits the number of processes/threads (prevents fork bombs).

Cgroups v1 vs v2

The Evolution:

Cgroups v1 (2008): Each controller has its own hierarchy. Complex and inconsistent.
Cgroups v2 (2016): Unified hierarchy. All controllers share the same tree. Simpler and more powerful.

Modern systems (systemd 226+) use v2 by default. Docker supports both.

Practical Example: CPU Limits

                
# Run a container with 50% of one CPU core

$ docker run -d --cpus="0.5" nginx

# Behind the scenes, Docker writes to cgroup files:

$ cat /sys/fs/cgroup/cpu/docker/<container-id>/cpu.cfs_quota_us

50000  # 50% of 100,000 microseconds

# Memory limit

$ docker run -d --memory="512m" nginx

$ cat /sys/fs/cgroup/memory/docker/<container-id>/memory.limit_in_bytes

536870912  # 512 MB in bytes

The OOM Killer

When a container exceeds its memory limit, the Linux kernel invokes the Out-of-Memory (OOM) Killer. It will kill processes in the container to free up memory.

                
# Stress test: allocate 600MB in a 512MB container

$ docker run --memory="512m" progrium/stress --vm 1 --vm-bytes 600M

[  123.456] Out of memory: Kill process 1234 (stress) score 999

[  123.457] Killed process 1234

Hands-On: Creating Cgroups Manually

You don't need Docker to use cgroups. Here's how to create one from scratch:

                
# Create a new cgroup for memory control

$ sudo mkdir /sys/fs/cgroup/memory/my_container

# Set memory limit to 100MB

$ echo 104857600 | sudo tee /sys/fs/cgroup/memory/my_container/memory.limit_in_bytes

# Add current shell to the cgroup

$ echo $$ | sudo tee /sys/fs/cgroup/memory/my_container/tasks

# Now this shell and all its children are limited to 100MB!

$ python3 -c "x = 'a' * (200 * 1024 * 1024)"  # Try to allocate 200MB

Killed

3. Union File Systems: The Layers

Why are Docker images so fast to pull? Because of Union File Systems (OverlayFS).

Instead of copying the entire OS every time, Docker uses layers. A "Container" is just a thin Read-Write layer on top of a stack of Read-Only layers (the Image).

Base OS (Ubuntu) Read-Only

App Dependencies Read-Only

Container Layer Read-Write

How Layers Work: A Dockerfile Example

Every instruction in a Dockerfile creates a new layer. Let's see this in action:

                
# Dockerfile

FROM ubuntu:20.04          # Layer 1: Base OS (80MB)

RUN apt-get update         # Layer 2: Package index (30MB)

RUN apt-get install -y nginx  # Layer 3: Nginx (50MB)

COPY app.conf /etc/nginx/  # Layer 4: Config file (4KB)

CMD ["nginx", "-g", "daemon off;"]  # Layer 5: Metadata only

                
# Build and inspect layers

$ docker build -t myapp .

$ docker history myapp

IMAGE          CREATED BY                                      SIZE

abc123         CMD ["nginx" "-g" "daemon off;"]                0B

def456         COPY app.conf /etc/nginx/                       4KB

ghi789         RUN apt-get install -y nginx                    50MB

jkl012         RUN apt-get update                              30MB

mno345         FROM ubuntu:20.04                               80MB

Copy-on-Write (CoW) in Action

The Magic: If your container wants to modify a file that exists in a lower read-only layer, the kernel copies that file up to the top Read-Write layer first. The original file remains untouched in the image. This is why spinning up 100 containers takes almost no extra disk space!

                
# Start a container

$ docker run -it ubuntu bash

# Modify a file from the base image

root@container:/# echo "modified" >> /etc/hosts

# The file is copied to the container's writable layer

# Original /etc/hosts in the image is unchanged

# Inspect the diff

$ docker diff <container-id>

C /etc

C /etc/hosts  # Changed

Layer Optimization: The Right Way

Bad Dockerfile practices create bloated images. Here's how to optimize:

❌ Bad: Multiple Layers

                        
RUN apt-get update

RUN apt-get install -y curl

RUN apt-get install -y vim

RUN rm -rf /var/lib/apt/lists/*

# Result: 4 layers, cache not cleaned

✅ Good: Single Layer

                        
RUN apt-get update && \

    apt-get install -y curl vim && \

    rm -rf /var/lib/apt/lists/*

# Result: 1 layer, cache cleaned

Multi-Stage Builds: The Secret Weapon

Build your app in one stage, copy only the binary to the final stage. Result: tiny production images.

                
# Stage 1: Build

FROM golang:1.20 AS builder

WORKDIR /app

COPY . .

RUN go build -o myapp  # Creates a 50MB binary

# Stage 2: Production (starts fresh!)

FROM alpine:3.18

COPY --from=builder /app/myapp /usr/local/bin/

CMD ["myapp"]

# Final image: ~10MB (Alpine + binary)

# No Go compiler, no source code!

Inspecting Layers with Dive

                
# Install dive (layer explorer)

$ docker run --rm -it \

  -v /var/run/docker.sock:/var/run/docker.sock \

  wagoodman/dive:latest myapp:latest

# Shows each layer's size and contents interactively

4. The Plumbing: Runtimes (runc & containerd)

Here is a shocker: Docker doesn't actually run containers.

Docker is just a high-level manager. When you type docker run, a complex chain of command occurs to actually create the process.

                
                        1. Docker Daemon (dockerd): The API server. It takes your command.
                    
                        2. containerd: The industry-standard container supervisor. It manages the
                        lifecycle (start, stop, pause) and image transfer.
                    
                        3. runc: The low-level runtime. This is the tool that actually
                        talks to the Kernel to create Namespaces and Cgroups.

Why the split? This modular architecture (OCI Standard) allows different tools to work together. You can replace Docker with Podman or Kubernetes, and they all use runc at the bottom to talk to the Linux Kernel.

5. Networking: The Invisible Wire

How does a container with its own isolated Network Namespace talk to the internet? Through a Virtual Ethernet Pair (veth).

Think of a veth pair as a tube with two ends.

Host Network (docker0 Bridge)

Container Network (eth0)

1. The Kernel creates a pipe. One end (veth123) is plugged into the Host's bridge (docker0).
2. The other end is moved inside the container's namespace and renamed to eth0.
3. The container sends packets to eth0, they travel through the pipe, pop out on the host bridge, and go to the internet using NAT (Network Address Translation).

Summary

So, what is a container?

It's a Process...
Restricted by Namespaces (Isolation)...
Throttled by Cgroups (Limits)...
Constructed by runc (Runtime)...
Connected by veth pairs (Networking).

There is no magic. Just clever use of Linux Kernel primitives.

Further Learning

Want to dive even deeper? Check out this excellent video tutorial:

Video Resource

Containers From Scratch
A hands-on tutorial showing you how to build containers from scratch using the Linux primitives we discussed.

Watch on YouTube →

Ready to Use Containers?

Now that you understand how containers work under the hood, learn how to actually use them with Docker.

Read: Docker - The Complete Guide →

The Great Illusion