hawkes.aiKey ideas
- The one idea — Attention + MLP + residual adds (+ a volume-leveler before each) = one repeatable block — talk, then think.
- How it is shown — The brick snapping together from its four parts, then running one full cycle on a token.
- The trap to avoid — Diagram intimidation — the famous architecture chart is these two moves, wired to the stream, nothing more.
That intimidating architecture diagram from every AI paper since 2017? Every box on it is something you can name in plain words: a library, a vault, a plus sign.
The one idea
Attention + MLP + residual adds (+ a volume-leveler before each) = one repeatable block — talk, then think.
Here's the LEGO brick the whole model is made of. Every part is one you already own. Watch it snap together. Move one: talk. The stream first passes a small leveling-ring — normalization, keeping the numbers in a healthy range so nothing blows up or fades out. Then the library: attention gathers what the room offers. Then the plus-stamp: the findings are added to the stream. Level, talk, add. Move two: think. Level again, then the vault — the MLP fires its detectors on the freshly-gathered context and writes the retrieved knowledge to the stream. Add.
How it works — the demo
The brick snapping together from its four parts, then running one full cycle on a token.
Done. One block, complete: talk, then think, each contribution stacked politely onto the conveyor. Nothing else is in there. And now the famous diagram — the one on every architecture paper that's been intimidating people since twenty-seventeen — collapses into a label. Attention box: the library. Feed-forward box: the vault. Those little curved arrows: the plus-stamps. The norm boxes: leveling-rings. Every box now has a face. There is nothing in the chart you haven't operated. Why this shape?
The trap to avoid
Diagram intimidation — the famous architecture chart is these two moves, wired to the stream, nothing more.
Why it matters — and what’s next
Because the brick's only interface is the stream: same-sized vector in, same-sized vector out. That's the whole contract. Which means bricks compose — stack two, stack ninety, the plumbing never changes. Modularity this clean is why the architecture scaled from lab toy to civilization-scale infrastructure without redesign. The trap: assuming every new model is new machinery. GPT-class, Llama-class, the frontier systems — under the branding, this same brick: talk, think, add, level. The genuinely interesting variations — mixture-of-experts vaults, shared-lens libraries — are edits to this brick, and when we meet them in the architecture act, you'll see them as renovations, not revolutions. So that's the brick — and one brick, honestly, is dumb. A little context-gathering, a little lookup. The magic is what happens when you stack it. Next episode: a hundred floors, and where "GPT" actually comes from.
GPT-class, Llama-class, the frontier systems: under the branding, it's the same repeating brick. Talk, think, add, level. The interesting variations are renovations, not revolutions.
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:30 of narration
Here's the LEGO brick the whole model is made of. Every part is one you already own. Watch it snap together.
Move one: talk. The stream first passes a small leveling-ring — normalization, keeping the numbers in a healthy range so nothing blows up or fades out. Then the library: attention gathers what the room offers. Then the plus-stamp: the findings are added to the stream. Level, talk, add.
Move two: think. Level again, then the vault — the MLP fires its detectors on the freshly-gathered context and writes the retrieved knowledge to the stream. Add. Done. One block, complete: talk, then think, each contribution stacked politely onto the conveyor. Nothing else is in there.
And now the famous diagram — the one on every architecture paper that's been intimidating people since twenty-seventeen — collapses into a label. Attention box: the library. Feed-forward box: the vault. Those little curved arrows: the plus-stamps. The norm boxes: leveling-rings. Every box now has a face. There is nothing in the chart you haven't operated.
Why this shape? Because the brick's only interface is the stream: same-sized vector in, same-sized vector out. That's the whole contract. Which means bricks compose — stack two, stack ninety, the plumbing never changes. Modularity this clean is why the architecture scaled from lab toy to civilization-scale infrastructure without redesign.
The trap: assuming every new model is new machinery. GPT-class, Llama-class, the frontier systems — under the branding, this same brick: talk, think, add, level. The genuinely interesting variations — mixture-of-experts vaults, shared-lens libraries — are edits to this brick, and when we meet them in the architecture act, you'll see them as renovations, not revolutions.
So that's the brick — and one brick, honestly, is dumb. A little context-gathering, a little lookup. The magic is what happens when you stack it. Next episode: a hundred floors, and where "GPT" actually comes from.