Design a distributed rate limiter handling 1,000,000 requests/sec across 100 API Gateway nodes. Compare Token Bucket, Fixed Window, Sliding Window Log, and.

A strong answer

A strong candidate frames Design a distributed rate limiter handling 1,000,000 requests/sec across 100 API Gateway nodes. Compare Token Bucket, Fixed Window, Sliding Window Log, and. around the exact memory layout, compiler semantics, and hardware execution model covered in [Distributed Rate Limiting: Token Bucket vs Sliding Window and Atomic Redis Lua Scripts]. Compare memory/accuracy tradeoffs across the 4 algorithms, detail the atomic Redis TIME + HMGET/HMSET Lua script, explain standard HTTP 429 Too Many Requests headers (Retry-After, X-RateLimit-Remaining), and scale past 100k QPS via Redis Cluster hash-slot sharding + local gateway batching.

Your API Gateway enforces a 100 requests / minute rate limit per tenant using a Fixed Window counter (INCR tenant:123:minute_42) and a naive GET -> if count < 100 -> INCR check in Redis. Two bugs hit in production: (1) A tenant sends 100 requests at 12:00:59.900 and another 100 requests at 12:01:00.100—pushing 200 requests in 200 milliseconds (2x the allowed rate!) across the minute boundary! (2) When 20 concurrent requests arrive at the same millisecond, all 20 gateway pods read GET == 99 before any pod runs INCR, letting all 20 through!

When engineers encounter this bottleneck in production, a surface-level API description rarely explains what went wrong. In real workloads, every line of code executes against concrete hardware constraints: fixed-width memory cells, CPU cache lines, kernel descriptor tables, network socket buffers, or accelerator memory buses. Without a clear mental model of Token Bucket vs Sliding Window Rate Limiting and Atomic Redis EVAL Lua Scripts, debugging degrades into guesswork—tuning timeouts, adding retries, or scaling instances blindly while the underlying architectural flaw remains untouched.

In a production interview, always connect the specification rule to its concrete runtime consequence: state what lives on the stack versus the heap or static storage segment, how the compiler or operating system validates alignments, lifetimes, and error codes, and what happens under adversarial or boundary inputs (such as zero-length buffers, integer overflow, concurrent mutation, or partial I/O). Backing your explanation with a small trace of register/memory state transitions immediately separates senior systems engineers from candidates who only memorized syntax.

What the interviewer is listening for

• Mastery of core invariant: Compare Fixed Window, Sliding Window Log, Sliding Window Counter, and Token Bucket rate limiters across accuracy and memory footprint.
• Mastery of core invariant: Eliminate distributed check-then-act race conditions by executing Token Bucket refill-and-deduct inside a single Redis Lua script.
• Mastery of core invariant: Scale rate limiting past single-node Redis limits using hash-slot sharding, clock-skew guards, and local in-memory fallback buckets.
• Hardware and ABI awareness: Explaining how sizes, alignments, calling conventions, or kernel boundaries behave on real 64-bit targets rather than relying on implementation-defined assumptions.
• Defensive engineering: Naming the exact compiler flags (-Wall -Wextra -Werror), static analyzers, or runtime sanitizers (-fsanitize=address,undefined,thread) that verify correctness in CI.

Common mistakes

• Explaining only surface syntax without tracing the underlying memory addresses, ownership lifetime, or kernel/user-space state transitions.
• Assuming undefined behavior (such as signed overflow, uninitialized reads, strict-aliasing violations, or dangling pointers) will "just work" because it passed an unoptimized -O0 debug build.
• Ignoring return codes, partial reads/writes, alignment padding, or cleanup paths when an allocation or syscall fails midway through a function.