The Runtime Theory
SystemInternalsdistributed systems

Idempotency: Retry with the Same Key, Dedupe Lookup, Replay Response

A step-by-step walk from the retried request with the same idempotency key to the dedupe lookup that replays the stored response instead of re-executing.

The Runtime Theory Team3 min read07 steps

layer stack

System

HWHardware
KKernel
RTRuntime
APPApplication
SYSSystem
CLIClient
NETNetwork
TLSCrypto
SRVServer

adjacent altitudes in this subsystem are still being traced

trace spine

  1. 01 client generates idempotency key
  2. 02 first attempt processed
  3. 03 response stored against key
  4. 04 client retries with same key
  5. 05 dedupe lookup hits
  6. 06 stored response replayed
  7. 07 key expires

Retries are the cause, idempotency is the cure. When a client retries a POST because the connection died, the server can't tell the retry from a new request — unless the request carries a key. This trace is what happens with the key.

trace stepSystem
Before the first request, the client creates a key: a UUIDv4 for a one-shot operation, or a stable business ID (order number, payment intent ID) for operations that are naturally unique. The key travels in the Idempotency-Key header. Crucially, a key must never be reused for a different request — that's the one rule that breaks everything.
trace stepSystem
The server does its dedupe lookup — GET idem:key in Redis, or SELECT ... WHERE key = ? — and finds nothing. This is the first time the key is seen, so the server executes the operation for real: the charge, the insert, the side effect. Alongside the execution it records the in-flight state (SET idem:key = processing, EX 24h), so concurrent duplicates can be rejected or parked.
trace stepSystem
The operation completes. The server stores the response — status code, headers, body — keyed by the idempotency key, with a TTL: Stripe uses 24 hours. The stored response is the memory of "this key was already satisfied." Storage cost: one small record per unique request, which is precisely the cost of the guarantee.
trace stepSystem
The client's connection timed out after the server executed but before the response arrived — the classic at-least-once failure. The client retries: same request, same key, same everything. From the server's perspective this is a new HTTP request that happens to carry a key it has seen.
trace stepSystem
The server's dedupe check finds the key — and here the state machine branches on which state: COMPLETED (the response is replayable) or IN_PROGRESS (the first execution is still running — return 409 Conflict or block-and-wait, per Stripe's behavior). If a second different request body arrives under the same key, it's a protocol violation: reject, don't execute.
trace stepSystem
The server returns the stored response verbatim — same 200, same body, same charge id — without touching the payment processor, the database row, or any side effect. The user is charged once; the client receives the answer it missed. The replay is the point: the retry became a cache read instead of a second execution.
trace stepSystem
After 24h the key record is evicted (Redis TTL) or archived. A retry that arrives after expiry is treated as a new request — which is correct only if the operation's semantics allow it (an idempotent payment might be re-runnable; a coupon redemption is not). The TTL is the window you promise: "retries within this window are safe; beyond it, re-execution is possible."
text
key = Idempotency-Key header (uuid or business id)
state = GET idem:{key}
if state == COMPLETED: return stored_response          # replay
if state == PROCESSING: return 409 or wait             # don't execute twice
# state == nil — first time:
SET idem:{key} = PROCESSING EX 24h
execute the operation                                   # side effects happen ONCE
SET idem:{key} = COMPLETED + response EX 24h
return response

The cost ledger: one Redis or DB read per request (including retries), one write on first execution, one stored response per unique key for 24h. In exchange, retries stop being dangerous: the client is free to retry forever, and every retry costs a cache hit instead of a duplicated side effect. Idempotency doesn't make at-least-once delivery better — it makes it safe to ignore.