hawkes.aiKey ideas
- The one idea — Attention is O(n²): double the tokens, quadruple the pair-work — the curve behind long-context pricing and a decade of engineering arms races.
- How it is shown — The grid swelling: 1,000 tokens ≈ a million pairs; 100,000 ≈ ten billion — per layer, per head.
- The trap to avoid — "Just make the window bigger" — the window is a budget you pay quadratically to widen; advertised ≠ usable.
- What it sets up — The engineering counterattacks.
A thousand-token prompt costs about a million pair-comparisons. A hundred thousand tokens? Ten billion. Per layer. Per head. Your prompt grows in a line; the work grows in a square.
The one idea
Attention is O(n²): double the tokens, quadruple the pair-work — the curve behind long-context pricing and a decade of engineering arms races.
Why does a long document cost so much more than a short one — more than seems fair? This curve. It's the most consequential shape in AI economics — and you already built everything needed to read it. The grid from last episode: every token scores against every token — n tokens, n-squared cells. And squares grow treacherously. Double the tokens: four times the cells. Ten times the tokens: a hundred times. The prompt grows in a line; the work grows in a square. That mismatch is the whole story. Put numbers on it. A thousand-token prompt — a few pages: about a million pair-scores. A hundred-thousand-token prompt — a hefty book: ten billion.
How it works — the demo
The grid swelling: 1,000 tokens ≈ a million pairs; 100,000 ≈ ten billion — per layer, per head.
And that's one layer, one head. Real models run dozens of layers, many heads each — every one paying its own copy of the square. The bill compounds before the first word of your answer exists. You've already felt this curve. Long-context requests sit in pricier tiers. That pause before a long document's first answer-word — the machine settling the square before it can speak. And windows have limits at all because somewhere on this curve, every budget dies. Once you see the square, a decade of engineering snaps into focus as one negotiation with it. FlashAttention reorganizes the memory traffic. The KV cache — nine episodes ahead — spares re-paying old columns. Architectures share lenses across readers to shrink the bill. Radical bets try replacing the square outright, at a price.
The trap to avoid
"Just make the window bigger" — the window is a budget you pay quadratically to widen; advertised ≠ usable.
Why it matters — and what’s next
The engineering counterattacks.
All of them are coming in this series. None abolish the square; they haggle with it. The trap: "just make the window bigger." Nothing about that word "just" survives the square — widening is paid quadratically, in compute, memory, and latency. And an advertised window is not automatically a window the model uses well. Advertised-versus-usable context is its own episode later. So carry the reframe: context is a budget, not a bucket. Every token you add pays rent against every token that follows it. Stuffing a window is a cost decision, and curation beats accumulation — a sentence the agent act inherits; its whole craft is deciding what deserves the square. That's the bill, and you can now read it at sight. Next: back inside the mechanism, for its final flourish — because one library visit was never going to be enough.
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:45 of narration
Why does a long document cost so much more than a short one — more than seems fair? This curve. It's the most consequential shape in AI economics — and you already built everything needed to read it.
The grid from last episode: every token scores against every token — n tokens, n-squared cells. And squares grow treacherously. Double the tokens: four times the cells. Ten times the tokens: a hundred times. The prompt grows in a line; the work grows in a square. That mismatch is the whole story.
Put numbers on it. A thousand-token prompt — a few pages: about a million pair-scores. A hundred-thousand-token prompt — a hefty book: ten billion. And that's one layer, one head. Real models run dozens of layers, many heads each — every one paying its own copy of the square. The bill compounds before the first word of your answer exists.
You've already felt this curve. Long-context requests sit in pricier tiers. That pause before a long document's first answer-word — the machine settling the square before it can speak. And windows have limits at all because somewhere on this curve, every budget dies.
Once you see the square, a decade of engineering snaps into focus as one negotiation with it. FlashAttention reorganizes the memory traffic. The KV cache — nine episodes ahead — spares re-paying old columns. Architectures share lenses across readers to shrink the bill. Radical bets try replacing the square outright, at a price. All of them are coming in this series. None abolish the square; they haggle with it.
The trap: "just make the window bigger." Nothing about that word "just" survives the square — widening is paid quadratically, in compute, memory, and latency. And an advertised window is not automatically a window the model uses well. Advertised-versus-usable context is its own episode later.
So carry the reframe: context is a budget, not a bucket. Every token you add pays rent against every token that follows it. Stuffing a window is a cost decision, and curation beats accumulation — a sentence the agent act inherits; its whole craft is deciding what deserves the square.
That's the bill, and you can now read it at sight. Next: back inside the mechanism, for its final flourish — because one library visit was never going to be enough.