The Runtime Theory

Load Balancer Request Flow: Health Checks, Backend Selection, and Draining

Follow a request through a load balancer — health probing, round-robin and hashing policies, connection pooling, X-Forwarded-For, and graceful drain.

The Runtime Theory Team09 stages

trace / request.md

DNS TO THE VIPLB TERMINATESCONNECTIONBACKEND CHOSENHEALTH CHECKSPRUNE POOLCONNECTIONTO BACKENDHEADERS REWRITTENRESPONSEFLOWS BACKIDEMPOTENT RETRYGRACEFUL REMOVAL

readyThe client resolves the service name to the load balancer's virtual IP (or an anycast address shared by a pool of balancers). The client never sees a backend address.

The load balancer's job is to make N backends look like one address. It terminates the client connection, picks a backend from the pool, and opens a second connection to it — which is why the client's connection and the backend's connection are separate, and why keep-alive pooling on the backend leg is the real performance lever.

Selection only happens among healthy backends: active probes and passive failure counters are what keep a dying server out of the rotation. The ordering of the flow matters — the decision uses the health state gathered before the request arrived, not during it. Hashing gives stickiness so sessions stay on one backend; least-connections spreads load when request costs vary. When a backend must leave, it drains: new requests stop, in-flight ones finish, and only then do its connections close. That is how a fleet can shrink without a single dropped request.