hawkes.aiKey ideas
- The one idea — Total = capacity and the memory bill; active = per-token compute and speed — an MoE computes like its active count but must be hosted like its total.
- How it is shown — The twin bars against real machinery: streaming speed tracking the small bar; the VRAM lease tracking the big one.
- The trap to avoid — "Runs like a 32B" hiding the 1T hosting bill — fits by total, streams by active.
- What it sets up — Who chooses the experts?
The announcement says it runs like a thirty-two billion model. True — for speed. Then you try to host it, and the full trillion parameters want a home in memory.
The one idea
Total = capacity and the memory bill; active = per-token compute and speed — an MoE computes like its active count but must be hosted like its total.
Every big model announcement carries two numbers — a trillion total, thirty-two billion active — and most readers pick one to believe. Wrong. Both are real; they just govern different bills. Learn which number rules what, and spec sheets stop fooling you. The big number — total — governs two things. Capacity: all the knowledge on all the shelves — what the trillion knows. And memory: every expert must live in VRAM, because any token might summon any of them, instantly. Episode ninety-five's lease, signed for the full mass. Sleeping experts pay full rent. The small number — active — governs per-token bills. Compute: each word does a thirty-two-billion model's math, not a trillion's. And bandwidth — episode ninety-two: only woken experts ride the conveyor, so it streams with a mid-sizer's gait.
How it works — the demo
The twin bars against real machinery: streaming speed tracking the small bar; the VRAM lease tracking the big one.
Speed and API pricing follow this bar. The sliver writes the freight bill. So amend episode one-oh-three's napkin, formally. Fits by total: memory follows the full mass — that's why trillion-scale models live on multi-GPU racks no matter how few experts wake. Streams by active: tokens-per-second follows the woken sliver. One model, two numbers, two lines of arithmetic. Write it in the margin forever. Now read the market with it. A giant total with a small active: deep knowledge, cheap per token — but a hosting burden only datacenters shoulder. A dense mid-sizer: one honest number doing both jobs — why your laptop prefers them. The comparison game changed: never weigh an MoE's total against a dense model's everything. Different bars, different sports.
The trap to avoid
"Runs like a 32B" hiding the 1T hosting bill — fits by total, streams by active.
Why it matters — and what’s next
Who chooses the experts?
The trap: half the sentence. "Runs like a thirty-two-B" is true — and the hosting reality is a trillion parameters of VRAM across a rack. Per-token cheap is not cheap to own; that gap is where budgets get surprised. The discipline is one question: both bars, please. Any pitch that shows one is hiding the other. One mystery remains, flickering at every chamber door: the gate. Smaller than a single expert's closet, it decides — token by token, floor by floor — where a trillion parameters' effort goes. Who trained it? What does it look at? What do experts really specialize in? Next: the router.
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:30 of narration
Every big model announcement carries two numbers — a trillion total, thirty-two billion active — and most readers pick one to believe. Wrong. Both are real; they just govern different bills. Learn which number rules what, and spec sheets stop fooling you.
The big number — total — governs two things. Capacity: all the knowledge on all the shelves — what the trillion knows. And memory: every expert must live in VRAM, because any token might summon any of them, instantly. Episode ninety-five's lease, signed for the full mass. Sleeping experts pay full rent.
The small number — active — governs per-token bills. Compute: each word does a thirty-two-billion model's math, not a trillion's. And bandwidth — episode ninety-two: only woken experts ride the conveyor, so it streams with a mid-sizer's gait. Speed and API pricing follow this bar. The sliver writes the freight bill.
So amend episode one-oh-three's napkin, formally. Fits by total: memory follows the full mass — that's why trillion-scale models live on multi-GPU racks no matter how few experts wake. Streams by active: tokens-per-second follows the woken sliver. One model, two numbers, two lines of arithmetic. Write it in the margin forever.
Now read the market with it. A giant total with a small active: deep knowledge, cheap per token — but a hosting burden only datacenters shoulder. A dense mid-sizer: one honest number doing both jobs — why your laptop prefers them. The comparison game changed: never weigh an MoE's total against a dense model's everything. Different bars, different sports.
The trap: half the sentence. "Runs like a thirty-two-B" is true — and the hosting reality is a trillion parameters of VRAM across a rack. Per-token cheap is not cheap to own; that gap is where budgets get surprised. The discipline is one question: both bars, please. Any pitch that shows one is hiding the other.
One mystery remains, flickering at every chamber door: the gate. Smaller than a single expert's closet, it decides — token by token, floor by floor — where a trillion parameters' effort goes. Who trained it? What does it look at? What do experts really specialize in? Next: the router.