hawkes.aiKey ideas
- The one idea — Two honest mechanisms collide: digit-chunk tokenization plus training-data neighborhoods where 9.11 genuinely "comes after" 9.9 (versions, chapters, dates).
- How it is shown — The flip: rephrase as explicit decimal math (or ask for the subtraction) and the answer often corrects — same weights, different neighborhood.
- The trap to avoid — Gotcha-benchmarking — one riddle, pass or fail, treated as an IQ verdict.
- What it sets up — Compensation is engineering you can't see.
The AI saying 9.11 is bigger than 9.9 isn't a glitch. In software versions, book chapters, and dates, 9.11 genuinely comes after 9.9. The machine learned from that text.
The one idea
Two honest mechanisms collide: digit-chunk tokenization plus training-data neighborhoods where 9.11 genuinely "comes after" 9.9 (versions, chapters, dates).
Ask which is bigger: nine point nine, or nine point eleven. Watch a machine that passes graduate exams flunk third grade. The infamous answer — "9.11 is bigger" — isn't a glitch. It's two honest mechanisms colliding. And you now own both of them. Force one: the chop. "Nine point one one" doesn't arrive as a quantity. It arrives as chunks — perhaps a nine, a dot, an eleven — and the model has no number line inside. Only text statistics about how digit-chunks behave. And eleven, as text, outranks nine.
How it works — the demo
The flip: rephrase as explicit decimal math (or ask for the subtraction) and the answer often corrects — same weights, different neighborhood.
Force two: the training data votes. Where does 9.11 live in text? Software versions — where it genuinely comes after 9.9. Book sections. Dates. In most of the text it learned from, nine-eleven really is later, higher, newer. The likeliest continuation follows the neighborhoods it knows. Here's the proof it's representation, not ability. Rephrase: "as decimal numbers, which is greater?" Or make it subtract them first. The answer often flips to correct.
The trap to avoid
Gotcha-benchmarking — one riddle, pass or fail, treated as an IQ verdict.
Why it matters — and what’s next
Compensation is engineering you can't see.
Same weights — the framing pulled the math neighborhood instead of the version-number neighborhood. The trap: the gotcha reflex. "It failed my riddle, so it's dumb" — and its twin, "it aced my riddle, so it's smart" — both misread one sample of dice. What you actually learned is sharper: how a question is tokenized and phrased selects which patterns answer it. That's a lever you now hold. Newer models mostly pass this now — more math data, reasoning training, or quietly calling a calculator tool. The blindness didn't vanish; it got compensated. Know the difference: compensation is engineering you can't see — and shouldn't blindly count on. One more tokenization crime scene to visit: a single invisible space that breaks working code. Then we finally leave the chopper — and meet meaning itself.
One phrase flips the famous 9.11 fail: ask 'as decimal numbers, which is greater?' and the answer often corrects. Same model, same weights — the framing picked the neighborhood.
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:30 of narration
Ask which is bigger: nine point nine, or nine point eleven. Watch a machine that passes graduate exams flunk third grade.
The infamous answer — "9.11 is bigger" — isn't a glitch. It's two honest mechanisms colliding. And you now own both of them.
Force one: the chop. "Nine point one one" doesn't arrive as a quantity. It arrives as chunks — perhaps a nine, a dot, an eleven — and the model has no number line inside. Only text statistics about how digit-chunks behave. And eleven, as text, outranks nine.
Force two: the training data votes. Where does 9.11 live in text? Software versions — where it genuinely comes after 9.9. Book sections. Dates. In most of the text it learned from, nine-eleven really is later, higher, newer. The likeliest continuation follows the neighborhoods it knows.
Here's the proof it's representation, not ability. Rephrase: "as decimal numbers, which is greater?" Or make it subtract them first. The answer often flips to correct. Same weights — the framing pulled the math neighborhood instead of the version-number neighborhood.
The trap: the gotcha reflex. "It failed my riddle, so it's dumb" — and its twin, "it aced my riddle, so it's smart" — both misread one sample of dice. What you actually learned is sharper: how a question is tokenized and phrased selects which patterns answer it. That's a lever you now hold.
Newer models mostly pass this now — more math data, reasoning training, or quietly calling a calculator tool. The blindness didn't vanish; it got compensated. Know the difference: compensation is engineering you can't see — and shouldn't blindly count on.
One more tokenization crime scene to visit: a single invisible space that breaks working code. Then we finally leave the chopper — and meet meaning itself.