hawkes.aiKey ideas
- The one idea — LoRA freezes the giant and learns the change as two thin matrices — because behavior change concentrates in a few directions — making customization workbench-cheap and swappable.
- How it is shown — The delta factored: a huge ΔW collapsing into thin A×B; training only the ribbons; merge-or-swap at serving; QLoRA putting a 70B on one GPU.
- The trap to avoid — Low-rank means low-capacity on purpose — style and task shape fit; deep new knowledge doesn't (the doors-episode law, now with its mechanism).
- What it sets up — Quantized bases (QLoRA).
Custom AI personalities ship like accessories — adapter files measured in megabytes, clipped onto one frozen giant and hot-swapped per customer. Here's the trick that makes them tiny.
The one idea
LoRA freezes the giant and learns the change as two thin matrices — because behavior change concentrates in a few directions — making customization workbench-cheap and swappable.
You can adapt a seventy-billion-weight giant by training something the size of a rounding error — and the reason why is one of the tidiest tricks in the stack. LoRA. Two thin matrices, one big idea. The reframe first. Don't touch the weights — learn the change. Freeze the original lattice, train a separate delta, and serve their sum: old weights plus adjustment. Which converts the problem into a question: can the adjustment — the delta — be stored cheaply? For behavior changes, the answer is a resounding yes. Now the trick. A full delta for one lattice would be millions of numbers — as big as what it adjusts.
How it works — the demo
The delta factored: a huge ΔW collapsing into thin A×B; training only the ribbons; merge-or-swap at serving; QLoRA putting a 70B on one GPU.
LoRA stores it factored: two thin matrices whose product regenerates the full-size effect. A four-thousand-square delta at rank sixteen: under one percent of the cells. Train only the thin pair, per lattice, and the giant learns your behavior through ribbons. Why should thin ribbons capture real change? Because low-rank suffices in practice: thin adapters match full fine-tuning — behavior concentrates in a few directions, not millions. Style is a handful of arrows. LoRA works because the world cooperated — meaning-as-direction, cashing out again. The serving story seals it. Merge the ribbons into the base and inference costs nothing extra. Or keep them separate and hot-swap: one frozen giant wearing different adapters per customer — a wardrobe of owned behaviors, each a file measured in megabytes.
The trap to avoid
Low-rank means low-capacity on purpose — style and task shape fit; deep new knowledge doesn't (the doors-episode law, now with its mechanism).
Why it matters — and what’s next
Quantized bases (QLoRA).
Personalities, shipped like accessories. One more gift: quantize the frozen base — act four teaches the squeeze — and the whole rig fits a single GPU. QLoRA put seventy-billion-scale customization on hobbyist desks. And the trap follows from the very mechanism: low-rank is low-capacity, on purpose. Style fits in a few directions; a library of new facts doesn't fit in ribbons. Episode eighty-five's law — knowledge wants the desk — now has its mechanical reason. So the giant is adapted, styled, owned. But no door — prompt, ribbon, furnace — changes when its knowledge ends. Everything it learned stops at a date, and past that date sits a cliff with a confabulation-shaped hole at the bottom. Next: the knowledge cutoff.
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:30 of narration
You can adapt a seventy-billion-weight giant by training something the size of a rounding error — and the reason why is one of the tidiest tricks in the stack. LoRA. Two thin matrices, one big idea.
The reframe first. Don't touch the weights — learn the change. Freeze the original lattice, train a separate delta, and serve their sum: old weights plus adjustment. Which converts the problem into a question: can the adjustment — the delta — be stored cheaply? For behavior changes, the answer is a resounding yes.
Now the trick. A full delta for one lattice would be millions of numbers — as big as what it adjusts. LoRA stores it factored: two thin matrices whose product regenerates the full-size effect. A four-thousand-square delta at rank sixteen: under one percent of the cells. Train only the thin pair, per lattice, and the giant learns your behavior through ribbons.
Why should thin ribbons capture real change? Because low-rank suffices in practice: thin adapters match full fine-tuning — behavior concentrates in a few directions, not millions. Style is a handful of arrows. LoRA works because the world cooperated — meaning-as-direction, cashing out again.
The serving story seals it. Merge the ribbons into the base and inference costs nothing extra. Or keep them separate and hot-swap: one frozen giant wearing different adapters per customer — a wardrobe of owned behaviors, each a file measured in megabytes. Personalities, shipped like accessories.
One more gift: quantize the frozen base — act four teaches the squeeze — and the whole rig fits a single GPU. QLoRA put seventy-billion-scale customization on hobbyist desks. And the trap follows from the very mechanism: low-rank is low-capacity, on purpose. Style fits in a few directions; a library of new facts doesn't fit in ribbons. Episode eighty-five's law — knowledge wants the desk — now has its mechanical reason.
So the giant is adapted, styled, owned. But no door — prompt, ribbon, furnace — changes when its knowledge ends. Everything it learned stops at a date, and past that date sits a cliff with a confabulation-shaped hole at the bottom. Next: the knowledge cutoff.