TRT
Sign in
easyExpertdesign

Walk through how `kube-scheduler` schedules a Pod (`PreFilter -> Filter -> PostFilter/Preemption -> Score -> Reserve -> Permit -> Bind`). How do you combine.

Explain the synchronous Scheduling Cycle (`AssumePod` in `Reserve`) vs asynchronous Binding Cycle, contrast `podAntiAffinity` (hard 1-per-node limit) vs `topologySpreadConstraints` (`maxSkew`), and pair multi-AZ spreading with `PDB` (`maxUnavailable: 1`).

TRT practice prompt — not a verified question from a named employer.

The Runtime Theory Team3 min read

A strong answer

Look inside kube-scheduler and the Eviction API: (1) The Two-Phase Scheduling Pipeline (Single-Threaded Scheduling Cycle + Concurrent Binding Cycle): For each unscheduled Pod in the Priority Queue, kube-scheduler takes a lock-free snapshot of its node cache and runs the synchronous Scheduling Cycle: PreFilter $\to$ Filter (Predicates: NodeResourcesFit, NodeAffinity, TaintToleration, PodTopologySpread—which nodes can run this Pod?) $\to$ PostFilter (Preemption: if 0 nodes fit, can evicting lower-priority Pods on a node make room without violating a PDB if possible?) $\to$ Score (Priorities: each plugin scores feasible nodes 0..100, multiplied by weight; highest total score wins!) $\to$ Reserve (optimistically marks resources as used in the in-memory cache!). Then the asynchronous Binding Cycle runs concurrently in a goroutine to write the Binding object to kube-apiserver! On a 5,000-node cluster, percentageOfNodesToScore stops filtering once ~10%–50% of feasible nodes are found so scheduling stays < 5 ms! (2) topologySpreadConstraints (maxSkew: 1): Setting topologySpreadConstraints across both topologyKey: topology.kubernetes.io/zone and kubernetes.io/hostname guarantees kube-scheduler never stacks all replicas on 1 node or 1 AZ! (3) PodDisruptionBudget (PDB) & the Eviction API: kubectl drain does not call raw DELETE /pods; it calls the POST /pods/<name>/eviction subresource, which atomically checks the PDB (disruptionsAllowed = currentHealthy - minAvailable). If replicas == 3 and minAvailable == 3 (or maxUnavailable: 0), disruptionsAllowed is 3 - 3 = 0, permanently blocking every voluntary node drain!

For any PodDisruptionBudget protecting a replica set with currentHealthy pods and threshold minAvailable (or desired - maxUnavailable), the Kubernetes Eviction API permits a voluntary pod eviction if and only if currentHealthy - minAvailable >= 1 (status.disruptionsAllowed >= 1). Because kube-apiserver processes Eviction requests using an optimistic-concurrency update on the PodDisruptionBudget status (disruptionsAllowed) and rejects with HTTP 429 Too Many Requests whenever currentHealthy <= minAvailable, concurrent kubectl drain operations across multiple nodes can never voluntarily reduce healthy replicas below minAvailable.

Using percentageOfNodesToScore (scoring a bounded sample of feasible nodes instead of all 5,000 nodes) keeps kube-scheduler throughput above 150–300 Pods/sec (~3 ms per Pod) even on 5,000-node clusters. In production systems (Kubernetes kube-scheduler Scheduling Framework, Karpenter node bin-packing & consolidation, Descheduler, and the Kubernetes policy/v1 PodDisruptionBudget & Eviction API.), engineers monitor these exact trade-offs to keep p99 latency predictable and prevent resource exhaustion under peak load.

What the interviewer is listening for

  • Traces how the kube-scheduler Framework (Filter -> Score -> Preempt -> Reserve -> Bind), Topology Spread, and PodDisruptionBudgets (PDB) operates in memory step by step rather than giving a vague textbook definition.
  • States the core correctness invariant and explains why edge cases do not violate it.
  • Quantifies time, space, and hardware costs (Filter + Score (percentageOfNodesToScore), topologySpreadConstraints (maxSkew: 1)) with concrete numbers.
  • Identifies real-world failure modes (Using requiredDuringSchedulingIgnoredDuringExecution podAntiAffinity on kubernetes.io/hostname for a 50-replica Deployment in a 20-node cluster) and explains how production systems mitigate them.

Common mistakes

  • Treating the kube-scheduler Framework (Filter -> Score -> Preempt -> Reserve -> Bind), Topology Spread, and PodDisruptionBudgets (PDB) as a black box without explaining the underlying memory or state transitions.
  • Ignoring worst-case tail latency or memory overhead (Creating a PodDisruptionBudget with minAvailable: 1 on a Deployment that has replicas: 1).
  • Applying the pattern without enforcing explicit capacity bounds or validation (Remembering that Kubernetes Scheduler only runs once at Pod creation time (IgnoredDuringExecution), so after nodes scale up and down over a week, Pods end up skewed (10 on Node A, 0 on Node B)).

This answer walks

Practice follow-ups

  1. 01Why does `PodDisruptionBudget` protect against **voluntary** disruptions (`kubectl drain`, Karpenter consolidation, spot rebalance eviction) but CANNOT prevent **involuntary** disruptions (a.
  2. 02What happens if a high-priority Pod triggers **Preemption** (`PostFilter`), evicts 2 lower-priority Pods on `Node-A`, and while the victims are terminating (`30s` grace period), another newly.

Not started

Sign in to save your learning progress.

Sign in to save