FlashAttention: the trick was memory.

The long AI conversations you take for granted were partly unlocked by a trick that invented zero new math. It just did the same steps in a smarter order.

The one idea

The famous speedup computes exact attention without ever writing the giant score-grid to slow memory — new bookkeeping, not new math.

One of the most famous breakthroughs in AI systems contains no new math. Same attention, same numbers, same exact answer — computed in a different order, in a different room. That reordering made it several times faster and helped unlock the long conversations you now take for granted. The problem: attention's score-grid. Every word against every word — double your context, quadruple the grid. Long documents made it the largest object the tower builds, far too big for the on-chip workbench. So it lived in the warehouse: computed, written out over the conveyor, read back for the next step. Episode ninety-three's law, violated at maximum scale. The trick: tiles. Cut the work into blocks small enough for the workbench. Score one block of words against another, fold the result into compact running totals, discard the tile, take the next.

How it works — the demo

Attention computed in tiles on the workbench, running totals carried, the big matrix never materializing anywhere.

Carry the totals to the end and you get the exact answer — while the giant grid never exists. Not compressed. Not approximated. Never born. The clever part is the bookkeeping. Attention's weighting normally needs to see all scores before it can balance them — episode thirty-five's blend. The trick keeps running totals that re-balance themselves as each tile arrives, so the final answer matches exactly. The paper's magic was proving the small-room schedule computes the very same attention. Bookkeeping, elevated to a breakthrough. What it bought: attention runs severalfold faster, and its intermediate memory stops growing quadratically. That helped move context windows from cramped pages toward the enormous ones you use today.

The trap to avoid

"Attention is quadratic, so long context is impossible" — know which wall (compute vs memory) actually binds before declaring limits.

Why it matters — and what’s next

Attention's other memory monster (the KV cache) gets its own tricks.

And the idea shipped everywhere — refined into successors, absorbed into every serious inference stack. When your hundred-page chat just works, this schedule is part of why. The trap: "attention is quadratic, therefore long context is impossible." The compute does still grow quadratically — that wall stands. But the wall that actually bound was memory traffic, and bookkeeping moved it. Before declaring anything impossible in this field, ask which wall you're facing. They fall by different tools, and more of them are movable than the pessimists assume. One more monster waits in the wings: the KV cache — episode forty-seven, the conversation's growing memory — hoarding warehouse space with every turn of your chat. Its legendary fix belongs to the serving group ahead. First, the blunter question that decides whether any of this runs on your machine at all: the warehouse itself. Next: why VRAM size decides everything.

This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.