TRT
Sign in
easyFoundationsconcept

Why does a Python list of numbers use so much more memory than a C array?

Explain references, boxed objects and contiguity, put numbers on the overhead, and say when it is worth switching to array or NumPy.

TRT practice prompt — not a verified question from a named employer.

The Runtime Theory Team1 min read
Learn it firstContiguous and scattered dataData either sits in one unbroken block, where an address can be computed, or in separate pieces joined by pointers, where it has to be followed. Most data structures are a choice between the two.

A strong answer

A C array of a million 32-bit integers is one contiguous block of 4 MB: the numbers themselves, side by side.

A Python list is also a contiguous block, but of references: 8-byte pointers to objects stored elsewhere on the heap. Every Python integer is a full object with a reference count, a type pointer and its digits, about 28 bytes for a typical value. So a list of a million distinct integers costs about 8 MB of pointers plus about 28 MB of integer objects: roughly 36 MB, nine times the C array. The list also keeps some spare capacity so that appends are cheap.

The layout costs speed as well as memory. Summing the list means following a pointer to a separate object for every element and checking its type each time. Summing a C array, or a NumPy array that stores raw numbers contiguously, is a tight loop over adjacent memory that the cache and prefetcher handle well, and that the compiler can vectorize.

When a program holds millions of numbers, use array.array or NumPy for compact storage, or NumPy for fast whole-array arithmetic. For a few thousand values, or mixed types, a list is fine.

What the interviewer is listening for

  • That a list holds references, and each integer is a separate object.
  • A rough calculation of the overhead.
  • The speed consequence (pointer chasing, type checks, no vectorization) as well as memory.
  • A practical alternative and when it is worth it.

Common mistakes

  • Saying Python lists are linked lists. They are dynamic arrays of references.
  • Quoting sys.getsizeof(list) as the total. It counts the pointer array, not the objects.
  • Forgetting that CPython caches small integers from −5 to 256, so a list of many small repeated values shares objects.

This answer walks

Practice follow-ups

  1. 01Why is a list of a million copies of the number 7 smaller than a list of a million different numbers?
  2. 02Does a Java ArrayList<Integer> have the same problem? What about an int[]?
  3. 03Besides memory, why is summing a NumPy array so much faster than summing a list?
  4. 04How would you measure the memory a list really uses?

Not started

Sign in to save your learning progress.

Sign in to save