hawkes.aiKey ideas
- The one idea — The vocabulary maps each token to an integer — the model's real input is a list of numbers, and the numbers themselves are arbitrary.
- How it is shown — A sentence collapsing to its ID list; the ID-as-parking-spot reveal.
- The trap to avoid — Assuming the ID carries meaning — token 9000 isn't "bigger" than token 9; meaning lives at the address, not in it.
- What it sets up — A menu slot with an unfilled table row = a ghost.
Every conversation you've ever had with an AI arrived at the door as a bare list of integers. Print one out sometime. It's a healthy shock.
The one idea
The vocabulary maps each token to an integer — the model's real input is a list of numbers, and the numbers themselves are arbitrary.
Behind every token is a number between zero and about two hundred thousand. That number is the whole story of this episode — and the door to everything after it. The menu is numbered — row zero, row one, up to the vocab size. Tokenize a sentence and the real output isn't chunks. It's the row numbers, in order. That list of integers is what the model receives. Nothing else survives the intake. "Hello world" might be two numbers. Your entire prompt — a shopping list of integers. Every conversation you've ever had with an AI was, at the door, exactly this: a list of numbers under two hundred thousand.
How it works — the demo
A sentence collapsing to its ID list; the ID-as-parking-spot reveal.
Print one out sometime — a whole conversation as bare integers. It's a healthy shock. And here's the twist: the numbers mean nothing. Token nine thousand isn't bigger, later, or better than token nine. IDs are parking spots, assigned by frequency and accident. Renumber the whole vocabulary and retrain — nothing about the model's behavior would change. The numbers are labels, not values. Which begs the real question. If the ID is arbitrary, where does meaning come from? Answer: each ID is an address — a row in a giant table the model learns.
The trap to avoid
Assuming the ID carries meaning — token 9000 isn't "bigger" than token 9; meaning lives at the address, not in it.
Why it matters — and what’s next
A menu slot with an unfilled table row = a ghost.
What's stored at the address is everything. The address itself is nothing. Hold both halves. A numbered menu, built by counting. A table of meanings, filled in by training. Two artifacts — and when a menu slot exists but its table row never got filled in… you get ghosts. That ghost story is next. Bring a flashlight.
Here's the twist nobody expects: the token numbers mean nothing. Token nine thousand isn't bigger or better than token nine. They're parking spots — meaning lives at the address.
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:15 of narration
Behind every token is a number between zero and about two hundred thousand. That number is the whole story of this episode — and the door to everything after it.
The menu is numbered — row zero, row one, up to the vocab size. Tokenize a sentence and the real output isn't chunks. It's the row numbers, in order. That list of integers is what the model receives. Nothing else survives the intake.
"Hello world" might be two numbers. Your entire prompt — a shopping list of integers. Every conversation you've ever had with an AI was, at the door, exactly this: a list of numbers under two hundred thousand. Print one out sometime — a whole conversation as bare integers. It's a healthy shock.
And here's the twist: the numbers mean nothing. Token nine thousand isn't bigger, later, or better than token nine. IDs are parking spots, assigned by frequency and accident. Renumber the whole vocabulary and retrain — nothing about the model's behavior would change. The numbers are labels, not values.
Which begs the real question. If the ID is arbitrary, where does meaning come from? Answer: each ID is an address — a row in a giant table the model learns. What's stored at the address is everything. The address itself is nothing.
Hold both halves. A numbered menu, built by counting. A table of meanings, filled in by training. Two artifacts — and when a menu slot exists but its table row never got filled in… you get ghosts.
That ghost story is next. Bring a flashlight.