hawkes.aiKey ideas
- The one idea — Some abilities appear to switch on abruptly at scale — and the skeptics showed discontinuous metrics can manufacture jumps from smooth progress; both papers are right about different things, and forecasting hangs on the gap.
- How it is shown — The same underlying progress plotted twice: all-or-nothing scoring producing a cliff-jump; partial-credit scoring producing a smooth climb.
- The trap to avoid — Certainty in either camp — "abilities switch on" and "emergence is a mirage" both overclaim; the ruler is part of every result.
Get eight digits right out of ten and you score zero. That grading quirk manufactured some of AI's spookiest moments — the ones called emergence.
The one idea
Some abilities appear to switch on abruptly at scale — and the skeptics showed discontinuous metrics can manufacture jumps from smooth progress; both papers are right about different things, and forecasting hangs on the gap.
Across the scaling era, researchers kept seeing the same eerie shape: an ability flat at zero, model after model — then, at one size, on. Arithmetic. Translation. Multi-step reasoning. "Emergence" became the era's spookiest word — until the skeptics checked the ruler. The act ends on the field's best fight. The claim, from the twenty-twenty-two survey: abilities absent below a scale — flat, model after model — then abruptly present above it, across families and tasks. The implication kept executives and safety researchers awake alike: if abilities switch on unannounced, every scale-up is a box you can't see into until you've paid. The skeptics' move was surgical: change the ruler. Exact-match gives zero credit for eight right digits of ten — smooth improvement scores zero, zero, zero… then everything.
How it works — the demo
The same underlying progress plotted twice: all-or-nothing scoring producing a cliff-jump; partial-credit scoring producing a smooth climb.
Re-measure with partial credit and the cliff melts into a climb. Verdict: many celebrated emergences were artifacts of all-or-nothing rulers. Who won? Both, about different exhibits. Under better rulers many cliffs melt — the surprise was in our scoring. But not all melt: some transitions hold under continuous measures, and you've met one mechanism that genuinely arrives in a burst — induction heads, snapping into place mid-training. Smooth rulers don't abolish phase changes; they make you prove them. And the fight is load-bearing. Smooth world: capabilities extrapolate — plan, price, prepare. Jumpy world: scale-ups carry surprises, and budgets and safety cases must hedge.
The trap to avoid
Certainty in either camp — "abilities switch on" and "emergence is a mirage" both overclaim; the ruler is part of every result.
Why it matters — and what’s next
The loss curve is smooth — episode seventy-one. Whether the abilities riding it are: that's this fight, live. The trap is certainty either way — both slogans over-claim. The durable lesson: the ruler is part of every result. When a headline says an ability emerged, your trained question: emergent — on what metric? Carry it into the evals act. That completes the training act — furnace, fuel, price, laws, finishing school, reasoning chambers, and the honest fights at the edge. You understand where minds come from. Act four descends to what carries them: silicon, memory walls, file formats, serving floors. Bring a hard hat.
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:30 of narration
Across the scaling era, researchers kept seeing the same eerie shape: an ability flat at zero, model after model — then, at one size, on. Arithmetic. Translation. Multi-step reasoning. "Emergence" became the era's spookiest word — until the skeptics checked the ruler. The act ends on the field's best fight.
The claim, from the twenty-twenty-two survey: abilities absent below a scale — flat, model after model — then abruptly present above it, across families and tasks. The implication kept executives and safety researchers awake alike: if abilities switch on unannounced, every scale-up is a box you can't see into until you've paid.
The skeptics' move was surgical: change the ruler. Exact-match gives zero credit for eight right digits of ten — smooth improvement scores zero, zero, zero… then everything. Re-measure with partial credit and the cliff melts into a climb. Verdict: many celebrated emergences were artifacts of all-or-nothing rulers.
Who won? Both, about different exhibits. Under better rulers many cliffs melt — the surprise was in our scoring. But not all melt: some transitions hold under continuous measures, and you've met one mechanism that genuinely arrives in a burst — induction heads, snapping into place mid-training. Smooth rulers don't abolish phase changes; they make you prove them.
And the fight is load-bearing. Smooth world: capabilities extrapolate — plan, price, prepare. Jumpy world: scale-ups carry surprises, and budgets and safety cases must hedge. The loss curve is smooth — episode seventy-one. Whether the abilities riding it are: that's this fight, live.
The trap is certainty either way — both slogans over-claim. The durable lesson: the ruler is part of every result. When a headline says an ability emerged, your trained question: emergent — on what metric? Carry it into the evals act.
That completes the training act — furnace, fuel, price, laws, finishing school, reasoning chambers, and the honest fights at the edge. You understand where minds come from. Act four descends to what carries them: silicon, memory walls, file formats, serving floors. Bring a hard hat.