Explain how a Kubernetes `ClusterIP` Service works under the hood in `iptables` mode, `IPVS` mode, and `eBPF` (`Cilium`) mode. Then explain the.

A strong answer

A Kubernetes Service ClusterIP (e.g., 10.96.0.10:80) is a purely virtual IP—it is not bound to any network card! Three generations of kernel packet translation route 10.96.0.10 to a real Pod IP (10.244.2.9:8080): (1) kube-proxy iptables Mode ($O(N)$ Linear Chain): Netfilter hooks every packet in PREROUTING/OUTPUT and walks a sequential linked list of KUBE-SVC-* rules ($O(N)$ in the number of Services!). Once matched, it picks one KUBE-SEP-* pod endpoint using --mode statistic --probability 0.333, performs Destination NAT (DNAT: 10.96.0.10 -> 10.244.2.9), and records the flow in the nf_conntrack connection-tracking table so reply packets from 10.244.2.9 get reverse-translated (SNAT back to 10.96.0.10). Updating 1 endpoint requires serializing and replacing the entire multi-megabyte iptables table! (2) kube-proxy IPVS Mode ($O(1)$ Hash Table): Uses the Linux kernel's L4 load balancer (IPVS) which matches (ClusterIP, Port) in an $O(1)$ hash table and supports Incremental API updates without rewriting all rules. (3) eBPF (Cilium cgroup/connect4 & sockops): Instead of letting the app create a TCP packet to fake IP 10.96.0.10, traversing TCP/IP + veth, and doing per-packet Netfilter DNAT, Cilium attaches an eBPF program directly to the connect() (sys_connect) system call! The instant the application calls connect(10.96.0.10:80), the eBPF program looks up 10.96.0.10:80 in an $O(1)$ eBPF Hash Map and rewrites the socket's destination struct in-place to 10.244.2.9:8080 before a single TCP SYN packet is even constructed—eliminating per-packet DNAT, SNAT, and iptables traversal completely!

With eBPF Socket-Level Load Balancing (cgroup/connect4), translation from (ClusterIP, ServicePort) to (BackendPodIP, TargetPort) occurs once per connection at connect() time ($O(1)$ per flow) rather than rewriting headers on every data packet ($O(P)$ per packet) in netfilter. Because the socket's sockaddr_in destination is mutated to (BackendPodIP, TargetPort) inside the cgroup/connect4 hook before tcp_v4_connect() builds the TCP control block (tcp_sock), all subsequent SYN, ACK, and data packets on that file descriptor carry the real BackendPodIP natively with zero per-packet NAT.

At 10,000 Kubernetes Services, switching from iptables ($O(N)$ linear scan + full-table iptables-restore) to eBPF (Cilium) reduces Service lookup from 10,000 rule checks to 1 eBPF Hash Map lookup (~50 ns) and speeds up endpoint sync by > 100x. In production systems (Cilium eBPF (kube-proxy replacement + sockmap), Linux kernel netfilter/iptables/nf_conntrack, Linux IPVS, and Kubernetes EndpointSlice controller.), engineers monitor these exact trade-offs to keep p99 latency predictable and prevent resource exhaustion under peak load.

What the interviewer is listening for

• Traces how Kubernetes ClusterIP Virtual IPs, iptables ($O(N)$) vs IPVS ($O(1)$) DNAT, conntrack, and eBPF (Cilium) Socket Load Balancing operates in memory step by step rather than giving a vague textbook definition.
• States the core correctness invariant and explains why edge cases do not violate it.
• Quantifies time, space, and hardware costs (kube-proxy iptables, kube-proxy IPVS) with concrete numbers.
• Identifies real-world failure modes (Closing your HTTP server's listening socket the instant SIGTERM arrives in a Kubernetes Pod) and explains how production systems mitigate them.

Common mistakes

• Treating Kubernetes ClusterIP Virtual IPs, iptables ($O(N)$) vs IPVS ($O(1)$) DNAT, conntrack, and eBPF (Cilium) Socket Load Balancing as a black box without explaining the underlying memory or state transitions.
• Ignoring worst-case tail latency or memory overhead (Making 10,000 outbound HTTP/DNS requests per second without HTTP Keep-Alive connection pooling (Connection: close)).
• Applying the pattern without enforcing explicit capacity bounds or validation (Using a single monolithic Kubernetes Service without Topology Aware Hints (service.kubernetes.io/topology-mode: Auto) across 3 AWS Availability Zones).