hawkes.aiKey ideas
- The one idea — When a conversation outgrows the window, the app trims — the model never forgets what it never saw; and even in-window content isn't equal (lost in the middle).
- How it is shown — Early messages sliding off the desk edge as the app trims from the front; the U-shaped recall curve across the window.
- The trap to avoid — Blaming the model for app-level trimming — and assuming everything in-window is equally visible.
- What it sets up — Trimming/summarizing policies = memory plumbing.
Your AI never forgot your chat. The app trimmed the beginning out of the transcript — and the model is never told anything was removed.
The one idea
When a conversation outgrows the window, the app trims — the model never forgets what it never saw; and even in-window content isn't equal (lost in the middle).
When a long chat stops recalling its own beginning, nothing got bored and nothing decayed. The beginning literally fell off the desk. Here's who pushed it — and it wasn't the model. Recall episode twenty: what reaches the model each turn is one assembled transcript. The application assembles it. So when the transcript outgrows the desk, the application decides what to cut — usually the oldest turns, sometimes swapped for a machine-written summary — and lays out what remains. The model is never told anything was removed. It answers from what's on the desk, sincerely, as if that were everything. So "it forgot" is the wrong verb. Forgetting implies once knowing.
How it works — the demo
Early messages sliding off the desk edge as the app trims from the front; the U-shaped recall curve across the window.
A trimmed beginning is a beginning this call's model never received — indistinguishable, from inside, from a conversation that started later. You remember the whole exchange; the machine holds a desk. And here's the finding that surprises even practitioners: presence on the desk doesn't guarantee visibility. Researchers planted facts at controlled positions in long contexts and measured recall — and it's U-shaped. Beginnings and endings: strong. The middle: measurably weak. "Lost in the middle" is real, replicated, and it means a two-hundred-thousand-token window is not two hundred thousand tokens of equal attention. The practical move falls straight out: position is a lever. Put instructions at the front. Put the thing you need used at the end, nearest the question.
The trap to avoid
Blaming the model for app-level trimming — and assuming everything in-window is equally visible.
Why it matters — and what’s next
Trimming/summarizing policies = memory plumbing.
Let bulk reference fill the middle. Same content, better geography — and measurably better answers. The trap: blaming the model for the butler's choices. Trimming policies are application decisions — invisible, various, and changing between products. The same model under two apps "remembers" differently because it's handed different desks. When an AI product forgets something, the diagnostic question is never "why is the model dumb?" It's: what actually reached the desk? So: the desk is finite, the butler trims, the middle sags — and you now read "forgetting" mechanically. Next group, we go under the desk, to the trick that makes chat economically possible at all — and the memory bill it quietly runs up. The KV cache. It's the best bargain in AI, and it eats GPUs alive.
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:30 of narration
When a long chat stops recalling its own beginning, nothing got bored and nothing decayed. The beginning literally fell off the desk. Here's who pushed it — and it wasn't the model.
Recall episode twenty: what reaches the model each turn is one assembled transcript. The application assembles it. So when the transcript outgrows the desk, the application decides what to cut — usually the oldest turns, sometimes swapped for a machine-written summary — and lays out what remains. The model is never told anything was removed. It answers from what's on the desk, sincerely, as if that were everything.
So "it forgot" is the wrong verb. Forgetting implies once knowing. A trimmed beginning is a beginning this call's model never received — indistinguishable, from inside, from a conversation that started later. You remember the whole exchange; the machine holds a desk.
And here's the finding that surprises even practitioners: presence on the desk doesn't guarantee visibility. Researchers planted facts at controlled positions in long contexts and measured recall — and it's U-shaped. Beginnings and endings: strong. The middle: measurably weak. "Lost in the middle" is real, replicated, and it means a two-hundred-thousand-token window is not two hundred thousand tokens of equal attention.
The practical move falls straight out: position is a lever. Put instructions at the front. Put the thing you need used at the end, nearest the question. Let bulk reference fill the middle. Same content, better geography — and measurably better answers.
The trap: blaming the model for the butler's choices. Trimming policies are application decisions — invisible, various, and changing between products. The same model under two apps "remembers" differently because it's handed different desks. When an AI product forgets something, the diagnostic question is never "why is the model dumb?" It's: what actually reached the desk?
So: the desk is finite, the butler trims, the middle sags — and you now read "forgetting" mechanically. Next group, we go under the desk, to the trick that makes chat economically possible at all — and the memory bill it quietly runs up. The KV cache. It's the best bargain in AI, and it eats GPUs alive.