This question is testing whether you understand autoscaling as a latency problem with capacity side effects, and whether you know the difference between a stable controller and an oscillator. A strong answer names the two clocks, explains hysteresis, and talks about what happens during the window where new capacity doesn't exist yet.
The mental model: two clocks run simultaneously. Reaction time — metric latency plus collection and decision intervals, often 1-5 minutes of pure delay — and provisioning time — instance creation, image pull, init, readiness, anywhere from 30 seconds to 5+ minutes. For a typical Kubernetes workload: 1 minute of metric lag, 15 seconds of HPA sync, 2 minutes of pod scheduling and readiness, roughly 3 minutes of exposure. During those 3 minutes the existing pods take the full spike — the metric that triggered the scale-out is the one degrading while you wait for the fix.
How the controller decides. HPA runs a fixed control loop every 15 seconds: desiredReplicas = ceil(currentReplicas * utilization / target). At 140% CPU against a 70% target it wants 2x the replicas, and scale-up is effectively one step — the stabilization window for scale-up is zero by default. Scale-down tolerates 5 minutes of low utilization before acting. That asymmetry is the whole point: scale-up races a latency cliff, and scale-down must be lazy, because a downscale that flips into an upscale the next minute is thrash — pods torn down, request queues drained, connection pools reset. Hysteresis, the gap between the threshold that triggers up and the threshold that triggers down, is what turns the controller into a stable system instead of an oscillator.
The cooldown trap. AWS Application Auto Scaling's default 300-second cooldown prevents flapping but means a 10x spike needs four scale events, 5 minutes apart, before the fleet catches up — the p99 has been red for 20 minutes. The fix hierarchy: scale on leading indicators — queue depth, request latency, synthetic probes react before CPU does; oversize the floor, minReplicas is a precomputed latency budget, not a safety net; shrink provisioning time with pre-warmed images, readiness gates that check real dependencies, and fast-start runtimes; and use predictive autoscaling where load is schedulable.
The scale-down cliff. Traffic drops, utilization drops, HPA scales down, a straggler request arrives at a pod being terminated, connection refused, retry storm, utilization spikes, scale up, thrash. The mitigations are graceful and slow termination: terminationGracePeriodSeconds, preStop hooks that drain in-flight requests, and readiness probes that remove pods from the service before the kubelet kills them.
Edge cases. Target 60% if the SLO tolerates 95% — the fleet overshoots the target by the margin of your reaction time. And autoscaling cannot fix a request that arrives before capacity exists: the real design question is whether the old fleet survives the worst-case gap between metric move and new capacity serving. If not, the answer is request shedding, queue depth limits, and latency-aware retry budgets — not autoscaling tuning.