Autoscaling Explained: The Controller Loop Behind Horizontal Scaling
Autoscaling is a control loop, not a feature. We walk through how the Horizontal Pod Autoscaler and AWS Auto Scaling actually operate: the metrics sampling window, the desired-replica calculation, cooldown and stabilization, and the feedback delay that makes naive CPU scaling oscillate. We show what happens under a real spike — request-rate scaling, custom metrics, queue depth — and why the controller's reaction time is governed by the metric window, not your will. The video ends with the failure modes: thrashing, scale-to-zero cold starts, and overshoot on slow-dequeue workloads.
Topics covered:
- The controller loop: observe, compare, act
- How desired replicas are computed from metrics
- Stabilization windows and why scaling oscillates
- Custom metrics, queue depth, and request-rate scaling
Related articles
Autoscaling Is a Latency Decision
Reaction time versus provisioning time, HPA sync intervals, cooldowns, and hysteresis — why autoscaling is a latency engineering problem before it is a capacity problem.
Kubernetes Scheduling Is a Packing Problem
How kube-scheduler bin-packs pods onto nodes with filter and score passes, why requests versus limits drive overcommit, and what QoS classes mean for eviction.
Containers Aren't Lightweight VMs
Namespaces, cgroups, seccomp, and the real isolation boundaries — what containers actually isolate and what they don't.
More in Cloud & Infrastructure
Edge Computing Explained: Where Compute Actually Sits
What edge computing actually is — the compute tiers from device to far edge to cloud, latency and bandwidth budgets, and which workloads genuinely benefit.
DetailsContainer Orchestration Basics: API Server, Controllers, and Scheduler
Container orchestration from first principles — what the API server, controller manager, and scheduler actually do, with Kubernetes as the working example.
DetailsThe Cost of Distributed Systems: Coordination, Consistency, and Failure
What distributed systems actually cost — coordination, consistency, and failure taxes — quantified with quorum math, tail latency, and retry-storm dynamics.
DetailsDNS and Traffic Routing in the Cloud: From Resolver to Anycast
How DNS actually routes traffic in the cloud — record resolution, CDN anycast, geo routing, load balancer handoffs, and how TTLs shape your failover story.
DetailsMulti-Region Architecture: Active-Active, Failover, and Replication
The real mechanics of multi-region deployments — where writes land, how replication propagates, what failover flips, and the latency math that constrains every design.
DetailsObject Storage Under the Hood: PUT, GET, and Erasure Coding
What happens inside an object store — the PUT and GET paths, metadata partitions, erasure coding, and why object storage is eventually consistent.
DetailsServerless Cold Starts, Measured: Where the Latency Actually Goes
Measured cold-start latency across Lambda, Cloud Functions, and container runtimes — what actually takes time and which optimizations genuinely reduce it.
DetailsKubernetes Scheduling Visualized: The Filter-Score Pipeline
What the Kubernetes scheduler actually does — how pending Pods become assigned to Nodes, the filter-then-score pipeline, and how taints and affinity shape placement.
DetailsContainer Images Explained: Layers, Manifests, and Digests
How container images are actually built and run — layers, manifests, content digests, and what the runtime does at pull and run time.
DetailsDepth, delivered weekly
One technical dispatch a week — articles and episode notes before they go public.
One technical dispatch per week. No noise.