TRT
Sign in
mediumFoundationsconcept

Why can't you get the nth character of a UTF-8 string in constant time?

Explain code points, variable-width encodings, how different languages represent strings, and the trade-offs between them.

TRT practice prompt — not a verified question from a named employer.

The Runtime Theory Team1 min read
Learn it firstStrings and encodingsA string is an array of numbers plus an agreement about what they mean. Unicode assigns the numbers, UTF-8 and UTF-16 turn them into bytes, and the choice decides what length and indexing mean.

A strong answer

Unicode gives each character a number, its code point. UTF-8 stores code points in 1 to 4 bytes: ASCII in one byte, most European and Middle Eastern scripts in two, most Asian scripts in three, and emoji and rarer characters in four. Because characters have different widths, there is no formula from character index to byte offset. Character n is at byte n only if every character before it was one byte. To find it, a program walks from the start, reading each leading byte to learn how long its character is. That is O(n).

The design is still a good trade. UTF-8 stores ASCII unchanged and compactly, has no byte-order issues, and resynchronizes from any position, because continuation bytes always start with the bits 10. Most programs read text sequentially, where variable width costs nothing.

Languages handle the indexing question differently. Go and Rust keep UTF-8 internally and index by byte; iterating gives code points. Java, JavaScript and C# use UTF-16, so indexing is O(1) by 16-bit unit, but characters outside the Basic Multilingual Plane, including most emoji, take two units. Python stores each string with 1, 2 or 4 bytes per character, chosen by its widest character, which gives O(1) indexing by code point. And what a user sees as one character can be several code points, a grapheme cluster, so even O(1) code-point indexing doesn't give "the nth visible character".

What the interviewer is listening for

  • Code points versus bytes, and the variable width of UTF-8.
  • The O(n) scan, and why UTF-8 is still the standard.
  • How at least two languages represent strings, and what their length counts.
  • Grapheme clusters as the next level of subtlety.

Common mistakes

  • Assuming one byte per character.
  • Saying UTF-16 is fixed-width.
  • Slicing UTF-8 at arbitrary byte offsets, which can cut a character in half.

This answer walks

Practice follow-ups

  1. 01How does Python manage O(1) indexing by code point without using 4 bytes for every character?
  2. 02Why is "😀".length 2 in JavaScript?
  3. 03What is a grapheme cluster, and when do you need to count those instead of code points?
  4. 04How would you truncate a UTF-8 string to at most 100 bytes without splitting a character?

Not started

Sign in to save your learning progress.

Sign in to save