hawkes.aiKey ideas
- The one idea — Truncation strategies decide which tokens are even eligible before the roll — three bouncers with three door policies.
- How it is shown — One market clipped three ways: count-based, mass-based, ratio-based; the dust-ring barred at the door.
- The trap to avoid — Dial soup — stacked extreme settings interact (order matters, and top-k=1 is greedy no matter the temperature); change one dial at a time.
- What it sets up — With the roll fully dialed, why do same prompts still differ?
Ever opened AI settings and seen top-p, top-k, min-p? They're bouncers — each one decides which words are even allowed on the wheel before it spins.
The one idea
Truncation strategies decide which tokens are even eligible before the roll — three bouncers with three door policies.
Temperature reshapes the market. These three decide who's even allowed in the room. Meet the bouncers — three door policies, three philosophies of exclusion. Bouncer one: top-k. The headcount policy — admit the fifty best-scored tokens, bar everyone else, rebuild the wheel from survivors. Simple, effective, and blunt: fifty is fifty whether the market has one obvious winner or three hundred worthy candidates. A fixed headcount ignores the shape of the crowd. Bouncer two: top-p — nucleus sampling, the industry's darling. The weighing policy: admit candidates until, together, they hold ninety percent of the probability, then close the door.
How it works — the demo
One market clipped three ways: count-based, mass-based, ratio-based; the dust-ring barred at the door.
Watch it breathe: on a confident market it admits three; on a torn one, sixty. The door adapts to the model's own certainty — which is exactly why it became the default. Bouncer three, the newest: min-p. The ratio policy: to board, a candidate must be at least — say — five percent as likely as the favorite itself. The bar scales with the leader: towering favorite, exclusive door; modest favorite, open house. Fans argue it handles high temperatures most gracefully — the dust never gets in, however wild the market. And now the system makes sense as a system. Temperature sets the adventure level; truncation sets the floor under it. Crank temperature with a good bouncer and you get wild-but-vetted — bold phrasing drawn only from candidates that earned the room.
The trap to avoid
Dial soup — stacked extreme settings interact (order matters, and top-k=1 is greedy no matter the temperature); change one dial at a time.
Why it matters — and what’s next
With the roll fully dialed, why do same prompts still differ?
That partnership, roughly moderate temperature plus top-p, is what most production systems actually run. The trap: dial soup. These settings interact — truncation and temperature compose in pipeline order, and extreme combinations do silly things: top-k of one is greedy no matter how high your temperature, because a wheel with one wedge has nowhere to spin. When output behaves strangely, return to defaults and move one dial at a time. Sampling is a system, and systems are debugged one variable at a time. Cockpit mastered: the knob, the bouncers, the partnership. Which leaves one honest mystery: set everything identically, ask the identical question twice — and still get two different answers. Next episode: every source of the dice, including the ones hiding in the hardware.
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:30 of narration
Temperature reshapes the market. These three decide who's even allowed in the room. Meet the bouncers — three door policies, three philosophies of exclusion.
Bouncer one: top-k. The headcount policy — admit the fifty best-scored tokens, bar everyone else, rebuild the wheel from survivors. Simple, effective, and blunt: fifty is fifty whether the market has one obvious winner or three hundred worthy candidates. A fixed headcount ignores the shape of the crowd.
Bouncer two: top-p — nucleus sampling, the industry's darling. The weighing policy: admit candidates until, together, they hold ninety percent of the probability, then close the door. Watch it breathe: on a confident market it admits three; on a torn one, sixty. The door adapts to the model's own certainty — which is exactly why it became the default.
Bouncer three, the newest: min-p. The ratio policy: to board, a candidate must be at least — say — five percent as likely as the favorite itself. The bar scales with the leader: towering favorite, exclusive door; modest favorite, open house. Fans argue it handles high temperatures most gracefully — the dust never gets in, however wild the market.
And now the system makes sense as a system. Temperature sets the adventure level; truncation sets the floor under it. Crank temperature with a good bouncer and you get wild-but-vetted — bold phrasing drawn only from candidates that earned the room. That partnership, roughly moderate temperature plus top-p, is what most production systems actually run.
The trap: dial soup. These settings interact — truncation and temperature compose in pipeline order, and extreme combinations do silly things: top-k of one is greedy no matter how high your temperature, because a wheel with one wedge has nowhere to spin. When output behaves strangely, return to defaults and move one dial at a time. Sampling is a system, and systems are debugged one variable at a time.
Cockpit mastered: the knob, the bouncers, the partnership. Which leaves one honest mystery: set everything identically, ask the identical question twice — and still get two different answers. Next episode: every source of the dice, including the ones hiding in the hardware.