hawkes.aiKey ideas
- The one idea — A GPU is three numbers — compute, VRAM capacity, memory bandwidth — and the famous one is usually the wrong one to watch.
- How it is shown — A cutaway with three gauges; different jobs making different gauges bind.
- The trap to avoid — Buying on TFLOPS for a job that's mostly moving weights — cores idle while the conveyor saturates.
- What it sets up — The cores mostly… wait.
People spend thousands on a GPU because of one number on the box — and for running AI, it's usually the least important of the three.
The one idea
A GPU is three numbers — compute, VRAM capacity, memory bandwidth — and the famous one is usually the wrong one to watch.
Crack a GPU open and it's a city of three districts, each with one gauge: how fast it can math, how much it can hold, how quickly it can move. Spec sheets shout the first number. For running AI, the other two usually decide everything. District one: compute. Thousands of cores, and among them the tensor cores — episode ninety's grid-eaters — doing trillions of operations a second. This is the number on the box, the one benchmarks brag about. It answers exactly one question: when the numbers have arrived, how fast can we math them? District two: VRAM — the warehouse. Memory stacked beside the cores, holding the entire model while it works: every weight, shelved and ready. Capacity is blunt: the workhorse datacenter card holds roughly eighty gigabytes, the newest near two hundred; gaming flagships, twenty-four to thirty-two. This gauge answers the most basic question there is: does your model even fit?
How it works — the demo
A cutaway with three gauges; different jobs making different gauges bind.
District three: bandwidth — the conveyor between warehouse and cores. Every weight that participates in a token must travel this road, and its width is measured in terabytes per second: a datacenter flagship moves several every second; a gaming card, about one. Hold that image of crates streaming to hungry furnaces. This quiet gauge is about to become the whole story. So which gauge matters? Depends on the question. Does the model fit at all? — the warehouse decides, full stop. How fast does it chat? — usually the conveyor, as the next episode proves. How fast does it crunch training math? — there, finally, the famous compute number reigns. Three questions, three different kings. The trap: shopping by the box number.
The trap to avoid
Buying on TFLOPS for a job that's mostly moving weights — cores idle while the conveyor saturates.
Why it matters — and what’s next
The cores mostly… wait.
For running models at home, TFLOPS is mostly decoration — the job is dominated by holding weights and moving them, so the warehouse and the conveyor set your experience. Buyers stare at the arithmetic gauge, then wonder why the expensive chip chats no faster. Fit first. Flow second. Math… usually third. And here's the uncomfortable scene the spec sheets don't show: during ordinary chat, those thousands of celebrated cores mostly stand idle — flashing to life as each load of weights arrives, then waiting for the road to deliver more. The machine isn't thinking most of the time. It's waiting. Next: the bottleneck nobody talks about.
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:45 of narration
Crack a GPU open and it's a city of three districts, each with one gauge: how fast it can math, how much it can hold, how quickly it can move. Spec sheets shout the first number. For running AI, the other two usually decide everything.
District one: compute. Thousands of cores, and among them the tensor cores — episode ninety's grid-eaters — doing trillions of operations a second. This is the number on the box, the one benchmarks brag about. It answers exactly one question: when the numbers have arrived, how fast can we math them?
District two: VRAM — the warehouse. Memory stacked beside the cores, holding the entire model while it works: every weight, shelved and ready. Capacity is blunt: the workhorse datacenter card holds roughly eighty gigabytes, the newest near two hundred; gaming flagships, twenty-four to thirty-two. This gauge answers the most basic question there is: does your model even fit?
District three: bandwidth — the conveyor between warehouse and cores. Every weight that participates in a token must travel this road, and its width is measured in terabytes per second: a datacenter flagship moves several every second; a gaming card, about one. Hold that image of crates streaming to hungry furnaces. This quiet gauge is about to become the whole story.
So which gauge matters? Depends on the question. Does the model fit at all? — the warehouse decides, full stop. How fast does it chat? — usually the conveyor, as the next episode proves. How fast does it crunch training math? — there, finally, the famous compute number reigns. Three questions, three different kings.
The trap: shopping by the box number. For running models at home, TFLOPS is mostly decoration — the job is dominated by holding weights and moving them, so the warehouse and the conveyor set your experience. Buyers stare at the arithmetic gauge, then wonder why the expensive chip chats no faster. Fit first. Flow second. Math… usually third.
And here's the uncomfortable scene the spec sheets don't show: during ordinary chat, those thousands of celebrated cores mostly stand idle — flashing to life as each load of weights arrives, then waiting for the road to deliver more. The machine isn't thinking most of the time. It's waiting. Next: the bottleneck nobody talks about.