hawkes.aiKey ideas
- The one idea — Over-aggressive quantization degrades models subtly and unevenly — fluency survives while the hardest skills crack, hidden from casual tests.
- How it is shown — A 2-bit model greeting flawlessly, then botching arithmetic its Q8 twin nails; average-score meters hiding the per-skill collapse.
- The trap to avoid — Judging a MODEL by someone's aggressive quant — half of "this model is dumb" is a review of the squeeze, not the brain.
- What it sets up — The shrunken brains travel as files.
An over-squeezed model greets you flawlessly, chats warmly, sounds completely fine — then quietly botches the math its full-size twin gets right. The damage hides where you don't test.
The one idea
Over-aggressive quantization degrades models subtly and unevenly — fluency survives while the hardest skills crack, hidden from casual tests.
Squeeze a model too hard and it doesn't break. That's the problem. It boots, it chats, it sounds like itself — and it's gotten subtly, selectively dumber. Over-quantization damage hides in the skills you test least. The dark side. What cracks first? The long tail. Multi-step math, careful code, rare languages, long-document recall — abilities held by delicate circuitry. What survives? Fluency — rehearsed across trillions of tokens, redundant. The greeting stays perfect while the spreadsheet formula dies — damage landing precisely where casual testing never looks. Why so uneven?
How it works — the demo
A 2-bit model greeting flawlessly, then botching arithmetic its Q8 twin nails; average-score meters hiding the per-skill collapse.
Outliers strain the press — one huge value stretches its block's shared scale, coarsening its neighbors. Abilities differ in redundancy: overlearned skills ride thick braided circuits; fragile ones hang on single threads. Average scores look decent while a specific thread has snapped. The report card hides the failing subject. So read the ladder honestly. Eight bits: safe. Five and four: the sweet spot where local AI lives. Three: compromises you'll notice. Two: heroics for the desperate. One wrinkle more — the press matters as much as the rung: a good four-bit quant can beat a careless five. The file's letters are a claim, not a guarantee. The defense is boring and works: test your task.
The trap to avoid
Judging a MODEL by someone's aggressive quant — half of "this model is dumb" is a review of the squeeze, not the brain.
Why it matters — and what’s next
The shrunken brains travel as files.
Run your actual prompts — the hard ones — against the pressed model and a reference, side by side. Trust the deltas, not the demo. Ten minutes of paired testing beats any leaderboard, because fractures are specific — only your workload knows where you stand. The trap infects half the discourse: judging a model from someone's aggressive quant. A two-bit pressing botches a task; the review says the model is dumb — but the squeeze was on trial, not the brain. When you benchmark, praise, or blame, name the pressing. Reading others' verdicts, ask which they ran. Most don't say. One last thing: all these pressed and pristine brains travel the world as files — downloaded by millions, from strangers. Questions this act hasn't asked yet: What's actually inside a model file? Why did one format need "safe" in its name? Next: the file trilogy — and the file that hacks you back.
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:30 of narration
Squeeze a model too hard and it doesn't break. That's the problem. It boots, it chats, it sounds like itself — and it's gotten subtly, selectively dumber. Over-quantization damage hides in the skills you test least. The dark side.
What cracks first? The long tail. Multi-step math, careful code, rare languages, long-document recall — abilities held by delicate circuitry. What survives? Fluency — rehearsed across trillions of tokens, redundant. The greeting stays perfect while the spreadsheet formula dies — damage landing precisely where casual testing never looks.
Why so uneven? Outliers strain the press — one huge value stretches its block's shared scale, coarsening its neighbors. Abilities differ in redundancy: overlearned skills ride thick braided circuits; fragile ones hang on single threads. Average scores look decent while a specific thread has snapped. The report card hides the failing subject.
So read the ladder honestly. Eight bits: safe. Five and four: the sweet spot where local AI lives. Three: compromises you'll notice. Two: heroics for the desperate. One wrinkle more — the press matters as much as the rung: a good four-bit quant can beat a careless five. The file's letters are a claim, not a guarantee.
The defense is boring and works: test your task. Run your actual prompts — the hard ones — against the pressed model and a reference, side by side. Trust the deltas, not the demo. Ten minutes of paired testing beats any leaderboard, because fractures are specific — only your workload knows where you stand.
The trap infects half the discourse: judging a model from someone's aggressive quant. A two-bit pressing botches a task; the review says the model is dumb — but the squeeze was on trial, not the brain. When you benchmark, praise, or blame, name the pressing. Reading others' verdicts, ask which they ran. Most don't say.
One last thing: all these pressed and pristine brains travel the world as files — downloaded by millions, from strangers. Questions this act hasn't asked yet: What's actually inside a model file? Why did one format need "safe" in its name? Next: the file trilogy — and the file that hacks you back.