hawkes.aiKey ideas
- The one idea — Saw every layer's matrices across GPUs — all cards compute their shard at once, stitching partial results every layer.
- How it is shown — One giant grid-meeting sawn in half; both halves multiplying simultaneously; the stitch; the chatter meter climbing.
- The trap to avoid — Tensor parallelism over slow links — the constant stitching turns a speedup into a slowdown.
- What it sets up — The cut with light chatter instead.
There's a way to split a model across GPUs that makes it faster — and over the wrong wiring, the exact same split makes it slower. The difference: chatter.
The one idea
Saw every layer's matrices across GPUs — all cards compute their shard at once, stitching partial results every layer.
First way to split a brain: the saw. Rip each layer's giant matrix down the middle — half the columns to this card, half to that. One token arrives; both cards fire at once. Tensor parallelism, the aggressive cut. How do the halves stay one mind? Why does a grid saw cleanly? Episode ninety: a matrix multiply is thousands of independent dot products. The columns never consult each other — so half the columns compute their share alone. The blade follows a seam that was always in the mathematics. Attention heads split just as naturally: different heads, different cards. The price is the stitch.
How it works — the demo
One giant grid-meeting sawn in half; both halves multiplying simultaneously; the stitch; the chatter meter climbing.
Each card ends the layer holding a partial answer; the halves must merge over the cables before the next floor starts. Every layer. Every token. The operation's name, met in every systems paper: the all-reduce. Tensor parallelism is a brilliant computation wrapped around a relentless conversation. All that stitching dictates the habitat. Inside one server, cards share specialized links built for this — the metronome hums. Across ordinary networks, each stitch would cost an eternity, and the speedup inverts. The rule: tensor parallelism lives inside a chassis — eight cards, one organism — and rarely leaves. What the saw buys: everything works, always. Every card holds an eighth of the weights and computes on every token — no idle floors, no waiting turns.
The trap to avoid
Tensor parallelism over slow links — the constant stitching turns a speedup into a slowdown.
Why it matters — and what’s next
The cut with light chatter instead.
Latency stays tight, which is why episode one-oh-four's eight-card serving chassis runs on this cut. When you chat with a frontier model, you're usually talking to a sawn brain. The trap: sawing over slow links. Consumer cards lack the specialized cables — and over an ordinary connection every layer's stitch crawls, cards idling mid-thought. The result can be slower than one card alone. Two-GPU home rigs want a different surgery — the next episode. But look at the tower again — there's a second seam, and it's gentler. Don't saw through the grids; cut between the floors. Each card takes whole layers, keeps its math private, and passes only a small result onward. Calm cables, simple pieces — and a brand-new problem called the bubble. Next: pipeline parallelism.
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:30 of narration
First way to split a brain: the saw. Rip each layer's giant matrix down the middle — half the columns to this card, half to that. One token arrives; both cards fire at once. Tensor parallelism, the aggressive cut. How do the halves stay one mind?
Why does a grid saw cleanly? Episode ninety: a matrix multiply is thousands of independent dot products. The columns never consult each other — so half the columns compute their share alone. The blade follows a seam that was always in the mathematics. Attention heads split just as naturally: different heads, different cards.
The price is the stitch. Each card ends the layer holding a partial answer; the halves must merge over the cables before the next floor starts. Every layer. Every token. The operation's name, met in every systems paper: the all-reduce. Tensor parallelism is a brilliant computation wrapped around a relentless conversation.
All that stitching dictates the habitat. Inside one server, cards share specialized links built for this — the metronome hums. Across ordinary networks, each stitch would cost an eternity, and the speedup inverts. The rule: tensor parallelism lives inside a chassis — eight cards, one organism — and rarely leaves.
What the saw buys: everything works, always. Every card holds an eighth of the weights and computes on every token — no idle floors, no waiting turns. Latency stays tight, which is why episode one-oh-four's eight-card serving chassis runs on this cut. When you chat with a frontier model, you're usually talking to a sawn brain.
The trap: sawing over slow links. Consumer cards lack the specialized cables — and over an ordinary connection every layer's stitch crawls, cards idling mid-thought. The result can be slower than one card alone. Two-GPU home rigs want a different surgery — the next episode.
But look at the tower again — there's a second seam, and it's gentler. Don't saw through the grids; cut between the floors. Each card takes whole layers, keeps its math private, and passes only a small result onward. Calm cables, simple pieces — and a brand-new problem called the bubble. Next: pipeline parallelism.