The Runtime Theory
Cloud & Infrastructure

Multi-Region Deployments and Latency

Speed-of-light RTT floors, active-passive versus active-active architectures, DNS-based routing, and replication lag — the real tradeoffs of running in multiple regions.

The Runtime Theory Team3 min read#multi-region#latency#dns#active-active#replication
On this page

A multi-region deployment is not a "redundancy feature." It is a bet against the physics and politics of distance: you are trading the consistency, complexity, and cost of keeping state synchronized across thousands of kilometers against the latency and availability you get in return. The first thing to understand is that the latency part is not negotiable — it is set by the speed of light in fiber, and no product decision changes it.

The physics floor

Information cannot move faster than light — and in glass fiber it moves at roughly two-thirds of vacuum speed:

text
c_vacuum  ≈ 299,792 km/s
c_fiber   ≈ 200,000 km/s  (index of refraction ~1.47)
 
one-way latency per 1,000 km ≈ 5 ms
round-trip latency ≈ 10 ms per 1,000 km

Real-world RTTs are always worse than the floor (router hops, queueing, cable great-circle routes vs. straight lines):

text
route                      ~distance    theoretical RTT    typical RTT
us-east-1 -> us-west-1     ~4,000 km    ~40 ms             ~65–70 ms
us-east-1 -> eu-west-1     ~5,500 km    ~55 ms             ~75–85 ms
us-east-1 -> ap-southeast-1 ~15,000 km   ~150 ms            ~220–250 ms
us-east-1 -> sa-east-1     ~7,700 km    ~77 ms             ~110–150 ms

Two facts fall out. First: a synchronous request that crosses the Atlantic in both directions pays ~150–170 ms of network time before your code runs. Second: 250 ms is perceptible — it is the difference between "instant" and "laggy" in user experience, and it is why the only honest argument for multi-region is bringing compute near the users, not "the cloud will make it fast."

Active-passive vs. active-active

The two architectures are different failure domains with different costs:

Active-passive (hot standby). One region serves all traffic; the other region runs the full stack idle (or a minimal replica), ready to take over. Traffic moves by failing over DNS or a global load balancer.

text
pros: simple consistency story — one writer, replicated to the standby
cons: failover is a human-orchestrated event (5–60 minutes, RTO decides)
      the passive region costs ~the same as the active one, serves 0% of users
      failover latency = detection + DNS TTLs + cache stampede at the new origin

Active-active (both regions serve). Traffic is split geographically — users in Europe hit eu-west-1, users in the US hit us-east-1. This is the only architecture where the latency benefit is real, and it is also where all the hard problems live: every region needs a copy of every write, and someone has to be authoritative for each piece of state.

text
pros: sub-100 ms latency for both user populations, true capacity doubling
cons: cross-region writes are slow (a write synced from eu-west-1 to us-east-1
      pays the same physics: +65–85 ms per replicated write)
      conflicts, split-brain, and "which region owns this key" are now your problem

DNS-based routing: how traffic gets there

Global load balancers (Route 53, Cloudflare, GCP LB) route by DNS policy. The classic pattern is latency-based routing — the DNS server answers the recursive resolver with the region that measured the lowest latency to that resolver's network:

json
{
  "AWSPolicy": {
    "Type": "latency",
    "Rules": [
      { "Region": "us-east-1", "Value": "api-east.example.com" },
      { "Region": "eu-west-1", "Value": "api-eu.example.com" }
    ]
  }
}
bash
dig api.example.com +short   # the resolver's network position decides which IP

Two properties make DNS routing slower to react than you'd like. First, TTL — clients and resolvers cache answers (30–300 s is common), so after a failover the stale answers keep sending traffic to the dead region for minutes. Second, DNS latency routing is per-resolver, not per-device: thousands of users behind one ISP resolver get routed as a herd, and the route can be wrong for a significant fraction of them (this is why CDNs build their own anycast/BGP routing instead of relying on DNS).

Replication: the actual multi-region cost

The database is the ceiling. Synchronously replicating writes across regions means every write waits for the far region's RTT — your write latency becomes the inter-region latency (add ~75 ms for a US→EU synchronous write). Asynchronous replication avoids that, but then you have replication lag (typically hundreds of ms to seconds under load, unbounded during network partitions) and a consistency model where a read may return stale data and a failover may lose the last seconds of writes.

text
synchronous:  write latency += inter-region RTT    consistency: linearizable
asynchronous: write latency unchanged              consistency: eventual, lag = f(load)

Aurora Global Database, Spanner, and CockroachDB represent the two ends: Spanner uses TrueTime clocks to make synchronous cross-region writes linearizable (you pay the RTT); Aurora Global makes writes local and replicates asynchronously (you pay with 1s-ish failover RPO). The choice is a PACELC decision — when the network partitions, do you favor Availability or Consistency — made before you pick the architecture, not after.

The most common real-world shape is active-active at the edge, active-passive at the core: stateless compute and static assets are active everywhere (that's the part multi-region is cheap for), while the authoritative database stays in one region, read replicas fan out to the others, and writes travel back. It captures most of the latency win for reads with a fraction of the complexity — and the complexity that remains (cache invalidation across regions, idempotency for replayed writes) is all in the state path.

The runtime view

  • Latency is physics: ~10 ms RTT per 1,000 km, typically 1.5–2x the theoretical floor in practice. No feature ships faster than the fiber.
  • Active-active is the only architecture that delivers the latency win, and it turns your database into the hard part.
  • DNS routing is herd-level and TTL-bound — failover is measured in DNS propagation time, not response time.
  • Replication is the real price: synchronous writes pay inter-region RTT; asynchronous writes pay lag and failover data loss.
  • The pragmatic shape: stateless everywhere, stateful once — edge active, core active-passive.