Event Loops vs Threads: How Concurrency Actually Gets Scheduled

"Handle 10,000 connections" has exactly two viable answers, and they are not philosophical preferences — they are two different contracts with the kernel. The thread-per-connection model gives each connection a thread that blocks on I/O. The event loop model gives all connections one thread that never blocks. Both exist because the alternative — thousands of blocked threads — is physically wasteful. Understanding the mechanics of each tells you which one your workload needs, and why Go ended up building a third model on top of both.

What a thread actually costs

A thread is not free state. On Linux, every thread gets an 8 MB user-mode stack (reserved virtual address space, committed page by page) plus a 16 KB committed kernel stack and a task_struct, and the scheduler must scan it on every tick. Context switches take ~1–3 microseconds of pure overhead — not counting the cache and TLB pollution that follows. Ten thousand threads means ~80 GB of virtual memory and a scheduler that spends most of its time switching between threads that are all, individually, waiting on one socket each. That is the C10K wall: not a hardware limit, a scheduling tax.

Blocking vs non-blocking I/O

The distinction is the entire subject. A blocking read() on a socket parks the thread in the kernel's wait queue and only returns when data arrives — which is exactly what makes thread-per-connection simple: your code reads like synchronous code, and the kernel does the waiting. A non-blocking socket returns immediately with EAGAIN when no data is ready. The thread never waits — which means someone else has to know when the socket becomes readable. That someone else is a readiness notification mechanism: select/poll (O(n) scan of every fd on every call), or on modern systems epoll (Linux) and kqueue (BSD/macOS), which maintain interest sets in kernel space and return only the ready events — O(ready), not O(total).

The event loop: one thread, no waiting

// The heart of every event loop: wait for readiness, dispatch, repeat
for (;;) {
    int n = epoll_wait(epfd, events, 1024, -1);   // sleep until something is ready
    for (int i = 0; i < n; i++) {
        handle(events[i].data.fd);                // non-blocking work, no I/O waits
    }
}

Every event-driven server you've used — nginx, Node.js, Redis — is this loop with different flavors of handle. The single thread registers interest in thousands of sockets, parks in epoll_wait (which is a blocking syscall, but blocks for all connections at once), and does work only when work exists. Blocking work still blocks it — which is why event loops route CPU-heavy jobs to worker threads and why they're terrible for embarrassingly computational loads.

Why 10,000 connections forces the choice

Ten thousand connections, thread-per-connection: ten thousand threads, ~80 GB of virtual address space, and a scheduler thrashing among them. Ten thousand connections on an event loop: one thread, one epoll interest set, and memory proportional to open sockets, not threads. The kernel never even notices the second design. Threads are the right tool when concurrency is low and each unit of work is blocking (I/O-heavy, latency-sensitive, simple math); event loops are the right tool when concurrency is high and work is mostly waiting. That's not a style choice; it's an accounting statement about stack memory and scheduling.

The fairness gap

Threads are preemptively fair: the kernel interrupts them every scheduling tick, so a runaway thread is eventually yanked. An event loop is cooperatively fair: one callback that computes for 500 ms freezes every other connection for 500 ms. This is the loop's structural weakness — there is no preemptor. Every serious event-loop design compensates with explicit yielding (splitting work into chunks), worker threads for CPU-bound paths, and careful instrumentation.

Go's M:N scheduler: both models at once

Go sidesteps the choice with goroutines: cheap logical threads multiplexed onto a pool of OS threads (the M:N model). When a goroutine does blocking I/O, the Go runtime rewrites that I/O through its netpoller — an internal event loop over epoll/kqueue — so the goroutine parks while the underlying thread is freed to run another goroutine. And when a goroutine makes a syscall that truly blocks, the runtime hands the thread to another goroutine and fetches a fresh one. You get thread-style blocking code with event-loop-style resource usage, because the scheduler sits between the two models:

for conn := range listener {          // thousands of these, one goroutine each
    go handle(conn)                   // runtime does the multiplexing
}

Production cost model and latency numbers

To apply Event Loops vs Threads: How Concurrency Actually Gets Scheduled effectively in production systems, engineers must move beyond qualitative diagrams and quantify the exact hardware and network costs at each boundary:

Failure modes, debugging, and operational checklist

In real incidents, systems rarely fail at average (p50) load—they fail when a slow dependency, skewed hot key, or resource leak pushes concurrency across a non-linear saturation cliff. When reviewing code or architecture that relies on Event Loops vs Threads: How Concurrency Actually Gets Scheduled, walk through three verification questions:

1. What is the invariant under partial failure or contention? If a worker crashes mid-operation, a network packet is delayed and reordered, or two threads race on the same state transition, prove which atomic primitive, version number, or idempotency key prevents silent corruption.
2. How is backpressure enforced when demand exceeds capacity? Unbounded in-memory queues, unthrottled retries, and blocking calls without explicit deadlines turn a localized 100 ms slowdown into a fleet-wide Out-Of-Memory (OOM) cascade. Always bound queue depths, propagate caller deadlines, and shed excess load early.
3. Which metrics and traces isolate the bottleneck in under 60 seconds? Instrument saturation, queue wait time vs service execution time, cache hit/miss ratios, and error rates by failure reason so on-call engineers can immediately distinguish a compute bottleneck from an I/O or lock-contention stall.