Winning Kaggle the Reproducible Way

Chapter 35

Memory Optimization for Large Datasets

Range is not precision: shrink a table only after checking what the smaller type destroys.

A blanket float32 downcast looks like free memory. One demonstration builds a purchase table, shrinks it two ways and measures both the megabytes saved and what survives: how many customer IDs stay distinct and how well a per-customer feature still predicts held-out rows.

Every example is constructed for teaching. Its numbers come from running the chapter activity's code, not from a competition result.

Demonstration 1 of 1

Blanket float32 against a checked downcast, by how large the IDs are

At what ID size does a blanket float32 downcast start merging customers, and how much memory does it save compared with a checked downcast?

Float32 keeps about seven significant digits. Below 2^24 every integer is exact; between 2^24 and 2^25 only even integers survive, so neighboring IDs merge in pairs, and each further doubling merges twice as many. Merged IDs make a per-ID aggregate describe the wrong customers. The same float32 is harmless for dollar amounts, whose error stays far below a cent. A checked downcast tests each column for what it would destroy.

Equation: Array bytes equal the number of values n times the bits per value b divided by eight, and the maximum conversion error is the largest absolute gap between each original value x i and its stored value x hat i

Scroll sideways for the whole equation

n is the number of values in a column, b the bits stored per value, x_i an original value and x-hat_i the value stored after conversion; e_max is the largest conversion error.

Predict first. IDs start at 2^24 (16.8 million) and span 39,756 customers. After a blanket float32 downcast, how many distinct customers remain?

Your prediction

Choose an example

Scroll sideways for the whole figure

Figure: Blanket float32 against a checked downcast, by how large the IDs are. Left: stacked memory bars for the unchanged, blanket float32 and checked tables when IDs start at 2^24; the checked table is 1.8 MB against 18.6 MB. Right: held-out R-squared and distinct IDs for each; blanket float32 keeps 19,933 IDs and R-squared 0.271.
Smallest customer ID, as a power of two: 2^24 (16.8 million)
Constructed example: a seeded synthetic purchase table; the memory figures are measured by the chapter activity.

Calculated values

Unchanged table
18.6 MB
Blanket float32
17.0 MB
Checked downcast
1.8 MB
Distinct IDs after blanket
19,933 of 39,756
R-squared, blanket
0.271
R-squared, checked
0.600

With IDs starting at 2^24: the table is 18.6 MB unchanged, 17.0 MB with blanket float32 (18.6 - 17.0 = 1.6 MB saved) and 1.8 MB checked (18.6 - 1.8 = 16.8 MB saved, ID stored as int32). 19,823 of 39,756 IDs vanish into a neighbor, and the held-out R-squared falls from 0.600 to 0.271. The checked table keeps all 39,756 IDs and R-squared 0.600. Amounts change by at most 0.003 cents.

Worked steps

  1. Memory saved by the blanket downcast: 18.6 - 17.0 = 1.6 MB.
  2. Memory saved by the checked downcast: 18.6 - 1.8 = 16.8 MB.
  3. Customers merged by float32: 39756 - 19933 = 19823.
  4. R-squared lost to the blanket downcast: 0.600 - 0.271 = 0.329.
  5. Measured peak while converting to the checked table: 18.6 + 11.4 = 30.0 MB (the original table plus the conversion's own peak allocation).

Use the idea

Before accepting any smaller dtype, compare distinct counts and task-unit errors, keep identifiers in an integer type wide enough for their range, and measure the table's memory before and after rather than assuming.

Where the conclusion applies

Constructed data on this machine's pandas. Memory is pandas deep usage of the resident table, so it depends on the pandas version and its string storage; the converted copy and the original are both alive during conversion, which is why conversion itself needs more memory than either table (its peak is measured with Python's tracemalloc allocation tracer, which sees pandas and NumPy buffers but not allocator overhead). The checked downcast adds an int32 range check and category strings, which are ordinary pandas practice rather than steps the chapter's helper contains.

Check your understanding: Why does the checked downcast save about 90% of the memory when the blanket float32 saves less than 10%, and why does it keep 64-bit IDs at 2^33?
In this table the long region strings hold most of the bytes (15.4 of 18.6 MB), so shrinking two numeric columns barely matters (17.0 MB) while storing the strings once as categories reaches 1.8 MB. The numeric columns still matter for correctness: IDs at 2^33 do not fit in int32 and merge in float32, so the check leaves them 64-bit and the table is 2.6 MB. A check that sometimes says no is the point.

Chapter 35 source: section "Downcast With an Information Check".