hawkes.aiKey ideas
- The one idea — Numeric formats trade per-weight exactness for size and speed — and neural networks need far less exactness than intuition says.
- How it is shown — The same number written on shrinking rulers — 32 bits of fine ticks down to 4 bits of coarse ones — cost and error both shrinking/growing in view.
- The trap to avoid — Equating precision with intelligence — past sufficiency, extra digits are just heat.
- What it sets up — Which formats do real models actually use?
Every number inside an AI can be written sloppier, with fewer digits, and the model barely cares. Past a point, precision isn't intelligence — it's just heat.
The one idea
Numeric formats trade per-weight exactness for size and speed — and neural networks need far less exactness than intuition says.
Episode twenty-seven: a weight is just a number. Here's the question nobody asks next: written down how precisely? Every number in the machine sits on a ruler — and how fine its tick marks are moves memory, speed, and billions of dollars. This is precision. How does a machine write a decimal? Three compartments. A sign. An exponent — the range dial, placing the value's neighborhood, tiny or astronomical. And a mantissa — the detail digits, pinpointing it within the neighborhood. The classic FP32 spends thirty-two bits being comfortably exact — the roomy handwriting computing grew up on. Now shrink the ruler. Sixteen bits: half the bytes, coarser ticks — each weight rounds to its nearest tick, carrying a whisper of error.
How it works — the demo
The same number written on shrinking rulers — 32 bits of fine ticks down to 4 bits of coarse ones — cost and error both shrinking/growing in view.
Eight bits: coarser again. Four: only a handful of representable values remain. And every halving pays triple — half the warehouse space, half the conveyor traffic, and on modern tensor cores, roughly double the matrix speed. Coarse handwriting is fast handwriting. Why doesn't coarseness wreck the brain? Because a weight doesn't stand alone — it votes, one voice in a blend of billions. Votes need a direction and a rough strength, not ten decimal places. And billions of tiny rounding errors, jittering in different directions, largely cancel in the blend. The crowd is robust to sloppy handwriting. Networks forgive — statistically. The industry has descended this staircase for a decade. Training moved from thirty-two bits to sixteen; the frontier now trains large parts in eight; and the newest datacenter chips execute four-bit matrix math natively — precision as a headline feature.
The trap to avoid
Equating precision with intelligence — past sufficiency, extra digits are just heat.
Why it matters — and what’s next
Which formats do real models actually use?
Every step down, the same silicon roughly doubles as a machine. Fewer bits isn't corner-cutting. It's the business model. The trap: equating precision with intelligence. An FP32 tower isn't smarter than its sixteen-bit twin — past sufficiency, extra digits buy nothing but heat and rent. The era's engineering skill is knowing where sufficiency sits — far lower than intuition says. Precision is a cost dial, not a quality dial — until, of course, you turn it too far. One subtlety remains, and it decided which format trains the frontier. Sixteen bits isn't one design — it's a budget you split between range and detail, and two famous formats split it opposite ways. One of them keeps training runs alive; the other falls off a cliff. Next: the working formats — BF16, FP16, FP8.
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:45 of narration
Episode twenty-seven: a weight is just a number. Here's the question nobody asks next: written down how precisely? Every number in the machine sits on a ruler — and how fine its tick marks are moves memory, speed, and billions of dollars. This is precision.
How does a machine write a decimal? Three compartments. A sign. An exponent — the range dial, placing the value's neighborhood, tiny or astronomical. And a mantissa — the detail digits, pinpointing it within the neighborhood. The classic FP32 spends thirty-two bits being comfortably exact — the roomy handwriting computing grew up on.
Now shrink the ruler. Sixteen bits: half the bytes, coarser ticks — each weight rounds to its nearest tick, carrying a whisper of error. Eight bits: coarser again. Four: only a handful of representable values remain. And every halving pays triple — half the warehouse space, half the conveyor traffic, and on modern tensor cores, roughly double the matrix speed. Coarse handwriting is fast handwriting.
Why doesn't coarseness wreck the brain? Because a weight doesn't stand alone — it votes, one voice in a blend of billions. Votes need a direction and a rough strength, not ten decimal places. And billions of tiny rounding errors, jittering in different directions, largely cancel in the blend. The crowd is robust to sloppy handwriting. Networks forgive — statistically.
The industry has descended this staircase for a decade. Training moved from thirty-two bits to sixteen; the frontier now trains large parts in eight; and the newest datacenter chips execute four-bit matrix math natively — precision as a headline feature. Every step down, the same silicon roughly doubles as a machine. Fewer bits isn't corner-cutting. It's the business model.
The trap: equating precision with intelligence. An FP32 tower isn't smarter than its sixteen-bit twin — past sufficiency, extra digits buy nothing but heat and rent. The era's engineering skill is knowing where sufficiency sits — far lower than intuition says. Precision is a cost dial, not a quality dial — until, of course, you turn it too far.
One subtlety remains, and it decided which format trains the frontier. Sixteen bits isn't one design — it's a budget you split between range and detail, and two famous formats split it opposite ways. One of them keeps training runs alive; the other falls off a cliff. Next: the working formats — BF16, FP16, FP8.