How does SCM_RIGHTS pass file descriptors between processes?

A strong answer

A strong candidate frames How does SCM_RIGHTS pass file descriptors between processes? around the exact memory layout, compiler semantics, and hardware execution model covered in Unix Domain Sockets and File Descriptor Passing. Explain ancillary data in sendmsg/recvmsg, how a receiving process gets a new descriptor number referring to the same open file description.

Throughout this module, we have evaluated inter-process communication mechanisms:

• Signals: Asynchronous interrupts with negligible payload.
• Pipes and FIFOs: Streaming bytes, but strictly unidirectional (requiring pairs of pipes for two-way communication).
• Shared Memory: Zero-copy memory access, but lacking built-in synchronization or notification.

Now consider modern security architectures: Privilege Separation and Sandboxing (used by OpenSSH, Chromium, Nginx, and systemd).

In a production interview, always connect the specification rule to its concrete runtime consequence: state what lives on the stack versus the heap or static storage segment, how the compiler or operating system validates alignments, lifetimes, and error codes, and what happens under adversarial or boundary inputs (such as zero-length buffers, integer overflow, concurrent mutation, or partial I/O). Backing your explanation with a small trace of register/memory state transitions immediately separates senior systems engineers from candidates who only memorized syntax.

What the interviewer is listening for

• Mastery of core invariant: Construct bidirectional stream and datagram channels using AF_UNIX and socketpair.
• Mastery of core invariant: Transmit open kernel file descriptors across process boundaries via sendmsg and SCM_RIGHTS.
• Mastery of core invariant: Parse ancillary control messages using CMSG_FIRSTHDR, CMSG_DATA, and CMSG_SPACE macros.
• Hardware and ABI awareness: Explaining how sizes, alignments, calling conventions, or kernel boundaries behave on real 64-bit targets rather than relying on implementation-defined assumptions.
• Defensive engineering: Naming the exact compiler flags (-Wall -Wextra -Werror), static analyzers, or runtime sanitizers (-fsanitize=address,undefined,thread) that verify correctness in CI.

Common mistakes

• Explaining only surface syntax without tracing the underlying memory addresses, ownership lifetime, or kernel/user-space state transitions.
• Assuming undefined behavior (such as signed overflow, uninitialized reads, strict-aliasing violations, or dangling pointers) will "just work" because it passed an unoptimized -O0 debug build.
• Ignoring return codes, partial reads/writes, alignment padding, or cleanup paths when an allocation or syscall fails midway through a function.