Interview prompt
Explain how a parser turns tokens into structure to an engineer who understands the surrounding system but has not used this technique. Walk from its contract to a concrete operation, then discuss where it fails or becomes expensive.
A strong answer
A lexer groups characters into tokens such as identifiers, numbers, and punctuation. A parser consumes that token stream according to a grammar and builds a structured representation, often an abstract syntax tree. Separating these jobs makes it easier to handle whitespace, literals, precedence, and syntax errors coherently.
For the expression 2 + 3 * 4, a precedence-aware parser constructs multiplication beneath addition so evaluation yields 14. Parentheses alter the tree by creating an explicit grouping. The AST usually omits surface details that do not affect meaning, such as redundant whitespace.
A complete answer also calls out the assumptions that control correctness. A parser accepting syntactically valid input does not prove the names are declared or types match; those checks belong to later semantic analysis. Error recovery matters in editors, where parsing should continue after a malformed statement to provide useful diagnostics.
Close by describing one representative test or measurement. Draw the AST for a - b - c under left-associative subtraction. Then explain what output would result if the parser accidentally made it right-associative.
Follow-up questions
Answer the follow-ups in the frontmatter. Use the linked article for the concept and the trace to make the explanation concrete.