hawkes.aiKey ideas
- The one idea — Three ways to change a model's behavior, ranked by cost and permanence — prompt first, LoRA when style must be permanent, full fine-tune rarely; none of them for facts.
- How it is shown — One behavior goal run through all three doors: the prompt achieving it in minutes; LoRA making it permanent; the furnace shown as overkill.
- The trap to avoid — Reaching for training when the context would do — and reaching for any of them to inject knowledge (retrieval's job).
- What it sets up — What LoRA actually is.
Before you pay to train an AI, know this: the three ways to change its behavior span five orders of magnitude in price — and the cheapest one usually wins.
The one idea
Three ways to change a model's behavior, ranked by cost and permanence — prompt first, LoRA when style must be permanent, full fine-tune rarely; none of them for facts.
You have a model, and you want it to behave differently. Three doors, priced across five orders of magnitude — and most teams walk through the wrong one first. Start at the lightest: often, you just ask. Door one: prompting. Write the behavior into the context — the casting call from episode seventy-four, plus a few worked examples on the desk. Minutes to try, instantly reversible, and more powerful than teams raised on older ML expect. The professional default: exhaust this door before touching weights. Door one's limits, honestly: the specification rides the desk, so you pay its tokens on every single call — behavior rented, never owned. Long specs can drift and wobble across sessions. When the instruction block outweighs the conversation, or consistency has to be contractual, renting stops making sense.
How it works — the demo
One behavior goal run through all three doors: the prompt achieving it in minutes; LoRA making it permanent; the furnace shown as overkill.
Door two: LoRA. Train a small add-on — the base model frozen, the adapter learning your behavior — then clip it on. Now the style is owned: no instructions renting desk-space, no per-call tax, consistency baked in, and adapters swap like lenses. Cost: hours on workbench hardware — via a trick elegant enough to earn the next episode. Door three: the full furnace — every weight re-forged. The deepest changes live here, and so do the costs: serious compute, serious data needs, and a real hazard — push hard on new behavior and old capabilities can dim, the forgetting problem. This door belongs to labs, domain overhauls, and distillation pipelines. For app teams: almost never — and "almost" is doing honest work. The tree, then. Format, persona, tone, task shape: prompt first, always — most journeys end there.
The trap to avoid
Reaching for training when the context would do — and reaching for any of them to inject knowledge (retrieval's job).
Why it matters — and what’s next
What LoRA actually is.
Style that must be permanent, consistent, and token-free at scale: LoRA. A deep domain rebuild with lab-grade resources: the furnace, rarely. And the branch that saves the most: facts route to none of these doors. Knowledge wants the desk; the desk's supply line is retrieval, act eight. The trap wears two costumes. Teams training to inject facts — the expensive confusion, again: weights are a terrible database, and retrieval embarrasses fine-tuning at that job. And teams skipping the prompt door out of engineering pride — as if asking nicely were beneath them, when it's the highest-ROI move in the stack. Next: inside door two — the two thin matrices that made adapting giants a weekend project. What LoRA actually does.
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:45 of narration
You have a model, and you want it to behave differently. Three doors, priced across five orders of magnitude — and most teams walk through the wrong one first. Start at the lightest: often, you just ask.
Door one: prompting. Write the behavior into the context — the casting call from episode seventy-four, plus a few worked examples on the desk. Minutes to try, instantly reversible, and more powerful than teams raised on older ML expect. The professional default: exhaust this door before touching weights.
Door one's limits, honestly: the specification rides the desk, so you pay its tokens on every single call — behavior rented, never owned. Long specs can drift and wobble across sessions. When the instruction block outweighs the conversation, or consistency has to be contractual, renting stops making sense.
Door two: LoRA. Train a small add-on — the base model frozen, the adapter learning your behavior — then clip it on. Now the style is owned: no instructions renting desk-space, no per-call tax, consistency baked in, and adapters swap like lenses. Cost: hours on workbench hardware — via a trick elegant enough to earn the next episode.
Door three: the full furnace — every weight re-forged. The deepest changes live here, and so do the costs: serious compute, serious data needs, and a real hazard — push hard on new behavior and old capabilities can dim, the forgetting problem. This door belongs to labs, domain overhauls, and distillation pipelines. For app teams: almost never — and "almost" is doing honest work.
The tree, then. Format, persona, tone, task shape: prompt first, always — most journeys end there. Style that must be permanent, consistent, and token-free at scale: LoRA. A deep domain rebuild with lab-grade resources: the furnace, rarely. And the branch that saves the most: facts route to none of these doors. Knowledge wants the desk; the desk's supply line is retrieval, act eight.
The trap wears two costumes. Teams training to inject facts — the expensive confusion, again: weights are a terrible database, and retrieval embarrasses fine-tuning at that job. And teams skipping the prompt door out of engineering pride — as if asking nicely were beneath them, when it's the highest-ROI move in the stack. Next: inside door two — the two thin matrices that made adapting giants a weekend project. What LoRA actually does.