Rate limiting protects services from overload. A token bucket allows bursts up to its capacity while constraining the average rate to the refill rate. Read the rate limiter practice to design the algorithm, and the load-balanced request trace to see where rate limiting sits on the critical path.
Stage-by-stage execution walkthrough
This request flow decomposes Rate Limiting with a Token Bucket into 5 sequential stages so you can trace where latency accumulates, which state transitions are irreversible, and where production failure modes occur.
Stage 1: Incoming request
The request arrives at the edge — a load balancer, API gateway, or the service itself. During this transition, the runtime enforces strict state invariants before handing control to the next stage: inputs are validated at the boundary, state mutations are ordered deterministically, and latency/resource costs are bounded so downstream consumers never observe a torn or partially initialized state.
Stage 2: Check token bucket
The limiter checks if a token is available. Each token represents one allowed request. During this transition, the runtime enforces strict state invariants before handing control to the next stage: inputs are validated at the boundary, state mutations are ordered deterministically, and latency/resource costs are bounded so downstream consumers never observe a torn or partially initialized state.
Stage 3: Consume token or deny
If a token is available, consume it and forward the request. If not, reject with HTTP 429. During this transition, the runtime enforces strict state invariants before handing control to the next stage: inputs are validated at the boundary, state mutations are ordered deterministically, and latency/resource costs are bounded so downstream consumers never observe a torn or partially initialized state.
Stage 4: Tokens refill over time
Tokens are added back at a fixed rate, up to the bucket capacity. This allows burst absorption. During this transition, the runtime enforces strict state invariants before handing control to the next stage: inputs are validated at the boundary, state mutations are ordered deterministically, and latency/resource costs are bounded so downstream consumers never observe a torn or partially initialized state.
Stage 5: Response returns
The response follows the same path back, or the client receives a rate-limit error. During this transition, the runtime enforces strict state invariants before handing control to the next stage: inputs are validated at the boundary, state mutations are ordered deterministically, and latency/resource costs are bounded so downstream consumers never observe a torn or partially initialized state.
Production invariants and failure modes
The client receives either a successful response or a 429, and can retry with exponential backoff. When debugging or capacity-planning this flow in production, instrument each stage boundary separately (measuring both p50 median and p99.9 tail latency), verify what happens when an intermediate stage times out or drops packets, and ensure retry or fallback paths preserve end-to-end idempotency.