How do function pointers work at the machine level, and how do C systems like the Linux kernel use struct vtables to implement polymorphism?

A strong answer

At the hardware level, a compiled C function is simply a sequence of machine instructions starting at a fixed virtual address in the .text segment. A function pointer is an 8-byte variable that stores that entry-point address, typed by the function's exact signature (return type and parameter types) so the compiler knows how to pass arguments in registers and read the return value. In int (*op)(int, int) = &add;, the parentheses around (*op) are mandatory because () binds tighter than *: int *op(int, int) declares a function returning int *, whereas int (*op)(int, int) declares a pointer to a function returning int.

Large C codebases—such as the Linux kernel's Virtual File System (struct file_operations), network device drivers, and SQLite's pluggable storage engines (sqlite3_vfs)—use structs of function pointers as explicit virtual method tables (vtables) to achieve runtime polymorphism in pure C:

struct transport_ops {
    ssize_t (*send)(void *ctx, const void *buf, size_t len);
    ssize_t (*recv)(void *ctx, void *buf, size_t len);
    void    (*close)(void *ctx);
};

A TCP transport and an in-memory mock transport each define a static const struct transport_ops populated with their own functions. Higher-level protocol code calls conn->ops->send(conn->ctx, buf, len) without knowing which implementation is attached.

What the interviewer is listening for

• Reads and writes function pointer syntax accurately (int (*cmp)(const void *, const void *)) and uses typedef to make callback APIs clean.
• Explains how pairing a const struct ops * vtable with an opaque void *ctx state pointer implements object-oriented interfaces in C.
• Notes the performance trade-off: a function pointer compiles to an indirect call (call *%rax), which prevents compile-time inlining unless the compiler can devirtualize it.

Common mistakes

• Omitting the parentheses in int (*fn)(int) and accidentally declaring a function prototype returning int *.
• Casting a function pointer to a different signature (e.g., casting int (*)(int *) to int (*)(const void *)) and calling it through the mismatched type—which is undefined behavior in ISO C (qsort comparators must accept const void * directly and cast inside the body).