hawkes.aiKey ideas
- The one idea — Unsupported formats get emulated — FP4 on older GPUs keeps the memory/bandwidth win but forfeits the doubled compute rate; know which wall you're on.
- How it is shown — An FP4 model on old vs new silicon: storage win surviving everywhere, the matmul-rate win appearing only where hardware speaks FP4.
- The trap to avoid — Buying a format's promise without checking your bottleneck — bandwidth-bound decode still profits from small weights; compute-bound work doesn't.
- What it sets up — Which promises are worth paying for at all.
Your GPU can run number formats it doesn't speak — it secretly unpacks every value wider, computes, repacks. The model runs. Half of what you paid for evaporates.
The one idea
Unsupported formats get emulated — FP4 on older GPUs keeps the memory/bandwidth win but forfeits the doubled compute rate; know which wall you're on.
Load a four-bit model on hardware that doesn't speak four-bit, and it runs — no warning. The file is tiny, the memory savings real. Meanwhile, at the tensor cores' door, every crate is quietly unpacked wider, computed at old speed, repacked. Half the promise, evaporating as steam. The fine print of what you actually bought: The law: formats without hardware lanes get emulated — unpacked into a width the chip speaks, computed there, repacked. Everything works; that's the mercy and the trap. "Runs" and "runs natively" are different products, and nothing on screen tells you which you got. The honest ledger. Surviving everywhere: the storage win — four-bit weights stored and hauled at four-bit width regardless. Quarter footprint, quarter conveyor traffic, the fit miracle intact. Lost without native lanes: the compute win — the doubled matrix rate lives only in silicon that speaks the format. Memory wins are portable.
How it works — the demo
An FP4 model on old vs new silicon: storage win surviving everywhere, the matmul-rate win appearing only where hardware speaks FP4.
Compute wins are not. Does the loss matter? Ask your wall — episode ninety-four's question, in hardware form. Home chat is decode: bandwidth-bound, and the surviving column is what it needed — small crates, fast hauls, quicker tokens despite the steam. Compute-bound work — long prefill, dense batches, training — bottlenecks exactly where the ghost lives. One mismatch, two verdicts. Know which wall your money hits. The match done right: on Blackwell-class silicon, four-bit crates flow straight through engines built for them — both wins alive, full promise. The formats carry episode ninety-eight's trick in hardware: per-block scales keeping four-bit honest. The pattern now: formats and silicon ship as couples. New format announced? Ask who speaks it.
The trap to avoid
Buying a format's promise without checking your bottleneck — bandwidth-bound decode still profits from small weights; compute-bound work doesn't.
Why it matters — and what’s next
Which promises are worth paying for at all.
The trap in the wild: benchmarks earned on native silicon, quoted beside a price tag from older silicon. Emulation is silent; the invoice is not. The ritual before any format-driven purchase: whose numbers? Which chip earned them? Which wall does my workload hit? Thirty seconds, thousands saved. You know the machines, the formats, the fine print. Now the question every team eventually asks: rent the tokens, or buy the machine? Not ideology — a break-even with one dominant variable, and most people guess the wrong one. Next: cloud versus self-host, with real arithmetic.
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:30 of narration
Load a four-bit model on hardware that doesn't speak four-bit, and it runs — no warning. The file is tiny, the memory savings real. Meanwhile, at the tensor cores' door, every crate is quietly unpacked wider, computed at old speed, repacked. Half the promise, evaporating as steam. The fine print of what you actually bought:
The law: formats without hardware lanes get emulated — unpacked into a width the chip speaks, computed there, repacked. Everything works; that's the mercy and the trap. "Runs" and "runs natively" are different products, and nothing on screen tells you which you got.
The honest ledger. Surviving everywhere: the storage win — four-bit weights stored and hauled at four-bit width regardless. Quarter footprint, quarter conveyor traffic, the fit miracle intact. Lost without native lanes: the compute win — the doubled matrix rate lives only in silicon that speaks the format. Memory wins are portable. Compute wins are not.
Does the loss matter? Ask your wall — episode ninety-four's question, in hardware form. Home chat is decode: bandwidth-bound, and the surviving column is what it needed — small crates, fast hauls, quicker tokens despite the steam. Compute-bound work — long prefill, dense batches, training — bottlenecks exactly where the ghost lives. One mismatch, two verdicts. Know which wall your money hits.
The match done right: on Blackwell-class silicon, four-bit crates flow straight through engines built for them — both wins alive, full promise. The formats carry episode ninety-eight's trick in hardware: per-block scales keeping four-bit honest. The pattern now: formats and silicon ship as couples. New format announced? Ask who speaks it.
The trap in the wild: benchmarks earned on native silicon, quoted beside a price tag from older silicon. Emulation is silent; the invoice is not. The ritual before any format-driven purchase: whose numbers? Which chip earned them? Which wall does my workload hit? Thirty seconds, thousands saved.
You know the machines, the formats, the fine print. Now the question every team eventually asks: rent the tokens, or buy the machine? Not ideology — a break-even with one dominant variable, and most people guess the wrong one. Next: cloud versus self-host, with real arithmetic.