How do you design a thread pool in C++ so task execution does not block submissions and shutdown never leaks or drops work?

A strong answer

A well-designed C++ thread pool separates queue coordination (which must hold the mutex) from task execution (which must never hold the queue mutex). Each worker runs an infinite loop: acquire std::unique_lock<std::mutex> lk(mtx_), wait on cv_.wait(lk, [&]{ return stop_ || !tasks_.empty(); }), check if stop_ && tasks_.empty() to return cleanly, move the front task into a local variable (auto task = std::move(tasks_.front()); tasks_.pop();), unlock lk, and only then invoke task(). If a worker ran task() while holding mtx_, the entire thread pool would degrade into a single-threaded queue!

For graceful shutdown in the destructor:

1. Lock mtx_ and set stop_ = true (so submit() rejects any new tasks and workers know no more tasks will arrive).
2. Unlock mtx_ and call cv_.notify_all() to wake all idle workers.
3. Loop over workers_ and call if (t.joinable()) t.join();. Because workers only exit when stop_ && tasks_.empty(), every previously queued job finishes before the destructor returns.

If callers need return values or exception propagation from tasks, wrap the callable in std::packaged_task<R()>, return its std::future<R> from submit(), and let the worker invoke the packaged_task—which automatically captures any thrown exception into the shared state rather than calling std::terminate on the worker thread.

What the interviewer is listening for

• Unlocks the queue mutex before executing task().
• Drains remaining tasks (stop_ && tasks_.empty()) before exiting the worker loop and joins all threads in the destructor (or uses std::jthread in C++20).
• Addresses what happens when a task throws an exception (catching or routing through std::packaged_task so std::thread does not call std::terminate()).

Common mistakes

• Holding the queue lock while executing task(), which serializes all workers and deadlocks if a task submits a child task to the same pool.
• Exiting the worker loop as soon as stop_ is true even when tasks_ still has pending items, silently dropping work on shutdown.