hawkes.aiKey ideas
- The one idea — When public test answers leak into training pools — usually without anyone lying — scores inflate silently; the durable defense is holding evaluation data close.
- How it is shown — The poisoned well: benchmark answers seeping from the public web into the training pile; the model "acing" what it memorized; canaries and fresh tests as partial fixes.
- The trap to avoid — Assuming a high public score reflects reasoning when it may reflect memorization — and assuming contamination requires bad actors.
AI models are trained on the public web. Benchmark answers are published on the public web. Do the math — that inflated score might be memory, not reasoning.
The one idea
When public test answers leak into training pools — usually without anyone lying — scores inflate silently; the durable defense is holding evaluation data close.
Here's how a benchmark gets poisoned without anyone lying. The test is published so people can use it — so its answers live on the public web. The training pile is the public web. The answers seep in, the model memorizes what it should reason through, the score inflates. Contamination — the poisoned well under the crisis. Why usually innocent? Scale — act three's training river, scraped from billions of pages. Among them, inevitably: benchmark pages, forums quoting questions, solution write-ups. Contamination enters by default — no one hand-inspects trillions of tokens. A systemic leak, not a moral failure — harder to fix than cheating, no villain to stop. What does it do to scores?
How it works — the demo
The poisoned well: benchmark answers seeping from the public web into the training pile; the model "acing" what it memorized; canaries and fresh tests as partial fixes.
It converts memorization into apparent reasoning. The model "solves" a problem it effectively saw, and the leaderboard can't tell recall from skill. The tell is novelty: a fresh problem of equal difficulty that couldn't have leaked collapses the score. That gap measures the contamination. Labs fight back, imperfectly. Canary strings — secret markers to detect training on test data. Decontamination sweeps that scrub known benchmark text. Fresh benchmarks, built after a cutoff. Each helps; none closes the gap, because paraphrases evade scrubbing and fresh tests age. An arms race, and the well keeps seeping. Which re-crowns last episode's lesson.
The trap to avoid
Assuming a high public score reflects reasoning when it may reflect memorization — and assuming contamination requires bad actors.
Why it matters — and what’s next
The durable defense is data that was never public: cases you kept close can't have leaked, by definition. Your private eval isn't just workload-matched — it's uncontaminated by construction. The act's two closing lessons are one: measure with your own reality, keep it yours. Two traps. A high public score means reasoning — when it may mean recall in disguise. Contamination means cheating — when it usually just means the web is enormous. The posture isn't outrage; it's vigilance: assume baseline leakage everywhere, discount accordingly, hold your clean data close. And that closes the model zoo: the committee architectures, the family tree, the two frontiers, the specialists, the benchmark wars — you can now read the entire landscape, and no leaderboard can fool you. Act six changes the sense entirely. Everything so far has been text. Next: the machine learns to see and hear.
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:30 of narration
Here's how a benchmark gets poisoned without anyone lying. The test is published so people can use it — so its answers live on the public web. The training pile is the public web. The answers seep in, the model memorizes what it should reason through, the score inflates. Contamination — the poisoned well under the crisis.
Why usually innocent? Scale — act three's training river, scraped from billions of pages. Among them, inevitably: benchmark pages, forums quoting questions, solution write-ups. Contamination enters by default — no one hand-inspects trillions of tokens. A systemic leak, not a moral failure — harder to fix than cheating, no villain to stop.
What does it do to scores? It converts memorization into apparent reasoning. The model "solves" a problem it effectively saw, and the leaderboard can't tell recall from skill. The tell is novelty: a fresh problem of equal difficulty that couldn't have leaked collapses the score. That gap measures the contamination.
Labs fight back, imperfectly. Canary strings — secret markers to detect training on test data. Decontamination sweeps that scrub known benchmark text. Fresh benchmarks, built after a cutoff. Each helps; none closes the gap, because paraphrases evade scrubbing and fresh tests age. An arms race, and the well keeps seeping.
Which re-crowns last episode's lesson. The durable defense is data that was never public: cases you kept close can't have leaked, by definition. Your private eval isn't just workload-matched — it's uncontaminated by construction. The act's two closing lessons are one: measure with your own reality, keep it yours.
Two traps. A high public score means reasoning — when it may mean recall in disguise. Contamination means cheating — when it usually just means the web is enormous. The posture isn't outrage; it's vigilance: assume baseline leakage everywhere, discount accordingly, hold your clean data close.
And that closes the model zoo: the committee architectures, the family tree, the two frontiers, the specialists, the benchmark wars — you can now read the entire landscape, and no leaderboard can fool you. Act six changes the sense entirely. Everything so far has been text. Next: the machine learns to see and hear.