hawkes.aiKey ideas
- The one idea — The whole model is one brick repeated — abstraction rises floor by floor, and there's no exotic machinery deeper in.
- How it is shown — The tower assembling; the same sentence's representation maturing floor by floor.
- The trap to avoid — Expecting a special "intelligence floor" — depth is repetition; the capability is cumulative.
- What it sets up — The towers differ in how tall and how wide.
Open floor ninety expecting exotic machinery. You find the same library and vault humming like floor three. There is no intelligence floor — the climb itself is the capability.
The one idea
The whole model is one brick repeated — abstraction rises floor by floor, and there's no exotic machinery deeper in.
One brick is dumb. A hundred stacked is GPT. This episode is that sentence, earned. And with that, you hold the complete recipe. Intake: text to tokens to vectors — act one. Then the same brick, repeated N times. Then one output device at the summit, still under wraps until the next group. That's the entire parts list of a large language model. Nothing is missing. Real heights: GPT-2 small ran twelve floors. Llama-3-seventy-B runs eighty. Frontier models: around a hundred, sometimes more.
How it works — the demo
The tower assembling; the same sentence's representation maturing floor by floor.
Twelve or a hundred and twenty — same brick, same two moves, different count. When a model card says "layers," it's counting these. What changes as the stream climbs? Abstraction. Early floors tend to sort out local structure — word forms, nearby grammar. Middle floors bind entities: who did what to whom. Late floors carry task-shaped information: what kind of text should come next. Tendencies, not assignments — nobody zoned the building — but measured again and again across models. The trap — episode five warned you, and now you can verify it: there is no intelligence floor. Open floor ninety expecting exotic machinery and you find the same library, the same vault, humming like floor three. The capability isn't in a penthouse. It's the accumulated re-description of a hundred modest floors — the climb itself.
The trap to avoid
Expecting a special "intelligence floor" — depth is repetition; the capability is cumulative.
Why it matters — and what’s next
The towers differ in how tall and how wide.
Why does repetition work? Compounding. Each floor's re-description is modest — gather a little context, retrieve a little knowledge. But each works on the output of all the floors below. A hundred modest improvements, compounding, produce sophistication no single floor contains. Depth is compound interest. One design question remains before we visit the summit. Towers can be tall — or they can be wide. Should a model be deep, or fat? It's a real tradeoff with real money attached. Next episode: what each buys.
GPT-2 small ran twelve layers. Frontier models run around a hundred. Same brick, same two moves, different count — and depth works like compound interest on understanding.
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:15 of narration
One brick is dumb. A hundred stacked is GPT. This episode is that sentence, earned.
And with that, you hold the complete recipe. Intake: text to tokens to vectors — act one. Then the same brick, repeated N times. Then one output device at the summit, still under wraps until the next group. That's the entire parts list of a large language model. Nothing is missing.
Real heights: GPT-2 small ran twelve floors. Llama-3-seventy-B runs eighty. Frontier models: around a hundred, sometimes more. Twelve or a hundred and twenty — same brick, same two moves, different count. When a model card says "layers," it's counting these.
What changes as the stream climbs? Abstraction. Early floors tend to sort out local structure — word forms, nearby grammar. Middle floors bind entities: who did what to whom. Late floors carry task-shaped information: what kind of text should come next. Tendencies, not assignments — nobody zoned the building — but measured again and again across models.
The trap — episode five warned you, and now you can verify it: there is no intelligence floor. Open floor ninety expecting exotic machinery and you find the same library, the same vault, humming like floor three. The capability isn't in a penthouse. It's the accumulated re-description of a hundred modest floors — the climb itself.
Why does repetition work? Compounding. Each floor's re-description is modest — gather a little context, retrieve a little knowledge. But each works on the output of all the floors below. A hundred modest improvements, compounding, produce sophistication no single floor contains. Depth is compound interest.
One design question remains before we visit the summit. Towers can be tall — or they can be wide. Should a model be deep, or fat? It's a real tradeoff with real money attached. Next episode: what each buys.