The Runtime Theory
mediumApplicationDSA#query-planning#performance#cost-model

What happens when a query is slow and the index is not the problem?

The interviewer is testing your debugging discipline beyond 'add an index' — reading the plan, bad cardinality estimates, spilling sorts, lock waits, and cache behavior.

The Runtime Theory Team2 min readasked at datadog · palantir · twilio

The trap in this question is the premise. Most people answer "add an index" — but the question says the index is not the problem, so the interviewer wants your process: how do you find out what the machine is actually doing?

First, get the plan and the real numbers: EXPLAIN (ANALYZE, BUFFERS) — or SET track_io_timing = on first. The single most common root cause is a cardinality estimate mismatch: the planner guessed 5 rows and the executor found 5 million. The planner chooses join orders and access paths from table statistics, and when stats are stale or the query's predicates are correlated, the plan is chosen for a query that doesn't exist. Check rows versus actual rows on every node; off by 1000x means the plan is wrong by construction, and no index will fix it. The fix is ANALYZE, extended statistics for correlated columns, or rewriting the predicate so the planner can reason about it.

Second, the index may be used and still be the wrong shape: an index scan that reads 40% of the table is slower than a seq scan, because it's random I/O to every page instead of sequential; the planner knows this (cost model) but its estimate was wrong. A sort can spill: work_mem exhaustion turns a memory sort into a multi-pass disk sort with temp files — a 10x slowdown. Hash joins spill the same way. Both show up in the plan as "Sort Method: external merge Disk".

Third — and this is the one most engineers forget — the slow query may not be executing at all. Lock waits look identical from the client side: the query hangs, pg_stat_activity shows wait_eventLock: relation or transactionid — and the fix is finding the long transaction holding the lock, not tuning SQL. Buffer pool eviction is the other invisible one: a query is fast in dev because everything is cached; in prod it's the first read after a checkpoint.

Finally, look outside the query: N+1 patterns across the app turn one slow query into thousands of fast ones; connection pool exhaustion makes everything slow identically; replication lag makes read replicas return stale data that gets retried and re-sent.

The senior framing: slow queries are a stack — planner estimates, executor mechanics, locks, cache, application behavior. You instrument first (pg_stat_statements, slow query log, EXPLAIN ANALYZE), then fix the layer the evidence points to. The index is one layer; the question exists because it's rarely the whole story.

This answer walks

Follow-ups they'll push on

  1. 01An index exists and the query still seq-scans. What do you check first?
  2. 02How do you tell whether time is spent planning, executing, or waiting for locks?
  3. 03Why would a hash join spill to disk and what does it cost?

More interviews in this topic

One dispatch a week

The trace behind each question, the tradeoff that explains it, and one technical dispatch per week — no noise.

One technical dispatch per week. No noise.