Containers Aren't Lightweight VMs

"Containers are lightweight VMs" is the most dangerous simplification in cloud computing. VMs provide hardware-level isolation through a hypervisor. Containers provide process-level isolation through kernel namespaces and cgroups. The difference is not a matter of degree — it is a matter of kind. A VM escape vulnerability is rare and catastrophic. A container escape is trivial and frequent, because containers share the host kernel. Understanding what containers actually isolate is the difference between using them correctly and having a security incident.

The VM vs. container architecture

VMs:                              Containers:
┌──────────────┐                 ┌──────────────┐
│   App        │                 │   App        │
│   Libs       │                 │   Libs       │
│   Guest OS   │                 ├──────────────┤
├──────────────┤                 │ Container    │
│  Hypervisor  │                 │ Runtime      │
│  (KVM/ESXi)  │                 │ (runc/crun)  │
├──────────────┤                 ├──────────────┤
│  Host OS     │                 │  Host OS     │
│  Hardware    │                 │  Hardware    │
└──────────────┘                 └──────────────┘
 
strong isolation:                weak isolation:
each VM has its own kernel       all containers share one kernel
hypervisor mediates all access   namespaces + cgroups enforce boundaries

The VM has its own kernel. The container does not. Every container on a host runs the same kernel. A kernel vulnerability in the host affects every container. A kernel vulnerability in a VM affects only that VM's guest kernel.

Linux namespaces: the isolation mechanism

Namespaces partition kernel resources so that each container sees its own isolated view:

The PID namespace is the most visible: inside a container, ps aux shows only the container's processes. The container's init process is PID 1. The host's processes are invisible. But the host can see all container processes — ps aux on the host shows every process in every container.

The network namespace gives each container its own IP address, its own routing table, and its own port space. Two containers can listen on port 80 simultaneously because they have different network namespaces. The container's port 80 maps to a different host port through Docker's port mapping (-p 8080:80).

cgroups: resource limiting, not isolation

Control groups (cgroups) limit resource usage — CPU, memory, I/O, network bandwidth. They do not provide isolation; they provide containment:

cgroup v2 limits:
  cpu.max:       100000 100000   # 1 CPU core (100ms per 100ms period)
  memory.max:    536870912       # 512 MB memory limit
  io.max:        8:0 rbps=1048576  # 1 MB/s read bandwidth
  pids.max:      256             # max 256 processes

When a container exceeds its memory limit, the OOM killer terminates it — the same mechanism that kills processes when the host runs out of memory. When a container exceeds its CPU limit, it is throttled — the kernel schedules it less frequently, not kills it.

The critical distinction: cgroups prevent a container from consuming more than its share of resources. They do not prevent a container from accessing resources it is not authorized to use. A container with CAP_NET_RAW can sniff network traffic even within its network namespace. A container with CAP_SYS_ADMIN can mount filesystems and potentially escape the container.

The real isolation boundary: capabilities and syscalls

The strongest isolation boundary in containers is not namespaces or cgroups — it is the syscall filter (seccomp-bpf) and Linux capabilities:

Linux capabilities split the root privilege into discrete units:

Docker drops most capabilities by default but retains CAP_NET_RAW, CAP_SYS_CHROOT, CAP_MKNOD, CAP_AUDIT_WRITE, and CAP_SETGID/CAP_SETUID. The --privileged flag grants all capabilities — a privileged container is effectively root on the host.

seccomp-bpf filters which syscalls a container can make. Docker's default seccomp profile blocks ~44 of ~300+ syscalls, including:

• mount — cannot mount filesystems
• reboot — cannot reboot the host
• ptrace — cannot trace other processes
• kexec_load — cannot load a new kernel
• open_by_handle_at — cannot access host filesystems via file handles

The seccomp filter runs on every syscall. It is the closest thing to a "firewall" between the container and the kernel.

What containers do NOT isolate

1. The kernel. All containers share the host kernel. A kernel vulnerability (e.g., Dirty Pipe, Dirty COW) affects every container on the host. VMs are immune because they run their own kernel.

2. The filesystem (by default). Containers share the host kernel's filesystem. The container's root filesystem is an overlay on the host filesystem. A container with CAP_SYS_ADMIN can mount the host filesystem and access any file.

3. The network (partially). Network namespaces isolate IPs and ports, but the host can see all container traffic. Containers on the same host can communicate through the host network if the Docker network allows it.

4. Time. Containers share the host's hardware clock. The time namespace (Linux 5.6+) allows per-container monotonic clocks, but wall-clock time is still shared.

5. Processes. The host can see and signal all container processes. docker exec enters a container's PID namespace from the host — it is not a remote connection, it is a namespace entry.

When containers are sufficient (and when they aren't)

Containers are sufficient when:

• You trust the code running inside the container.
• You are not running untrusted workloads.
• You need fast startup, dense packing, and simple orchestration.
• Security is layered (network policies, RBAC, image scanning).

Containers are NOT sufficient when:

• You run untrusted code (multi-tenant platforms, serverless).
• You need hardware-level isolation (financial services, regulatory compliance).
• You need to protect against kernel vulnerabilities.
• You need strong side-channel resistance (Spectre, Meltdown).

For untrusted workboxes, use gVisor (Google's application kernel that intercepts syscalls and implements them in userspace) or Kata Containers (lightweight VMs that run a minimal kernel per container, combining VM isolation with container semantics).

What this means for your code

1. Do not rely on containers for security isolation. Containers are process isolation, not hardware isolation. If your threat model requires hardware isolation, use VMs.

2. Drop capabilities aggressively. Start with --cap-drop ALL and add only what you need. Every capability is a potential escalation path.

3. Use the seccomp profile. Docker's default seccomp profile blocks dangerous syscalls. Do not disable it (--security-opt seccomp=unconfined) unless you have a specific reason.

4. Run as non-root. A container running as root (UID 0) has root's capabilities inside the container. Use a non-root user in the Dockerfile (USER 1000).

5. Scan images for vulnerabilities. A container image with a vulnerable library is an attack surface. Use Trivy, Grype, or Snyk to scan images before deployment.