Design a thread pool with a task queue.

A strong answer

A strong candidate frames Design a thread pool with a task queue. around the exact memory layout, compiler semantics, and hardware execution model covered in High-Throughput Worker Pools and Task Queues. Discuss worker threads, condition variable signaling, graceful shutdown, and work-stealing versus central queue trade-offs.

When building network services or compute pipelines, programs must process an ongoing stream of incoming tasks: HTTP requests, disk write blocks, or image transformation tiles.

This model works during low-volume testing, but collapses under production load:

1. Thread Creation Latency: Calling pthread_create() requires mmap for an 8 MB stack and a clone() system call (~15–30 microseconds). Creating a thread for a 50-microsecond task spends substantial CPU time in setup overhead.
2. Memory Exhaustion: If 5,000 clients connect simultaneously, allocating 5,000 thread stacks requires 40 gigabytes of virtual memory address space. The system fails with EAGAIN or ENOMEM.
3. Scheduler Thrashing: When hundreds of threads are runnable, the kernel Completely Fair Scheduler (CFS) switches between them rapidly. CPU cores spend more time saving and restoring registers and invalidating L1/L2 caches than executing user code.

In a production interview, always connect the specification rule to its concrete runtime consequence: state what lives on the stack versus the heap or static storage segment, how the compiler or operating system validates alignments, lifetimes, and error codes, and what happens under adversarial or boundary inputs (such as zero-length buffers, integer overflow, concurrent mutation, or partial I/O). Backing your explanation with a small trace of register/memory state transitions immediately separates senior systems engineers from candidates who only memorized syntax.

What the interviewer is listening for

• Mastery of core invariant: Eliminate thread creation overhead using persistent worker thread pools.
• Mastery of core invariant: Construct thread-safe bounded circular task queues using mutexes and condition variables.
• Mastery of core invariant: Implement clean multi-threaded shutdown protocols using broadcast wakeups and pthread_join.
• Hardware and ABI awareness: Explaining how sizes, alignments, calling conventions, or kernel boundaries behave on real 64-bit targets rather than relying on implementation-defined assumptions.
• Defensive engineering: Naming the exact compiler flags (-Wall -Wextra -Werror), static analyzers, or runtime sanitizers (-fsanitize=address,undefined,thread) that verify correctness in CI.

Common mistakes

• Explaining only surface syntax without tracing the underlying memory addresses, ownership lifetime, or kernel/user-space state transitions.
• Assuming undefined behavior (such as signed overflow, uninitialized reads, strict-aliasing violations, or dangling pointers) will "just work" because it passed an unoptimized -O0 debug build.
• Ignoring return codes, partial reads/writes, alignment padding, or cleanup paths when an allocation or syscall fails midway through a function.