The Runtime Theory
hardApplicationDSA#networking#concurrency#queues#load-balancing

Why does your API get slower at high concurrency even with plenty of CPU?

Probes whether you understand where real contention lives — sockets, threads, queues, and the kernel — when headroom is measured as idle cores, not bottlenecks.

The Runtime Theory Team2 min readasked at stripe · amazon · uber · meta

The trap here is "plenty of CPU" — the interviewer wants you to enumerate every queue and lock between the NIC and the response, because latency degrades long before utilization hits 100%. This is a Little's Law question dressed up as a performance question.

The mental model: latency = time in queue + time in service, and queues form everywhere.

Start with Little's Law: L = λW. At 1000 req/s with 200ms average response time, you need ~200 requests in flight. All 200 are simultaneously contending for: socket buffers, thread-pool slots, DB connections, and locks. As concurrency climbs, time-in-service stays flat but time-in-queue grows superlinearly. The system is "loaded" well before the CPU is.

Where the queues form:

  1. Threads. A thread-per-request model with a 200-thread pool: at 200 in-flight requests, the queue is full — request 201 waits for a thread, and its wait time adds directly to observed latency. Even with an async runtime, the task scheduler's run queues and its timers add scheduling latency when saturated.
  2. Kernel sockets. Each connection owns receive and send buffers. With many in-flight requests, the kernel's receive queue and the epoll ready lists grow; the TCP stack's softirq processing (see the receive path) contends for the same cores the app runs on — at ~1GB/s of copy work at line rate, memory bandwidth and cache-line contention become the ceiling before core count does.
  3. Locks and shared state. A single shared counter, an in-memory cache with a mutex, a connection pool with a contended semaphore: lock acquire times triple or quadruple under saturation as cache lines ping-pong between cores. A lock that takes 1µs uncontended can take 20-50µs contended — invisible in a profile at 10% utilization, dominant at 80%.
  4. Downstream dependencies. The API itself is fine; the DB connection pool (typically 50-100 connections) becomes the true bottleneck. Requests queue at the pool, and queue time is latency. 500ms DB queries × 50 connections caps you at ~100 req/s of DB throughput — no amount of spare CPU changes that.
  5. The accept/connection path. Epoll wakeups, accept() contention on the listen socket, and connection establishment under SYN backlog pressure add ms-scale waits that compound per request.

How you'd prove it: measure with utilization at the dependency (pool utilization, queue depth at the DB, softirq CPU%), then plot p50/p99 latency vs concurrent load — latency rises linearly with queue depth exactly where a pool saturates. Fixes follow the bottleneck: connection reuse and HTTP keep-alive (fewer handshakes, fewer sockets), bounded queues with rejection, pooling with accurate limits, and batching to raise service rate per connection.

This answer walks

Follow-ups they'll push on

  1. 01How would you confirm the bottleneck with a profile?
  2. 02What's the difference between latency going up and throughput going flat?
  3. 03How does connection reuse change this picture?

More interviews in this topic

One dispatch a week

The trace behind each question, the tradeoff that explains it, and one technical dispatch per week — no noise.

One technical dispatch per week. No noise.