hawkes.aiKey ideas
- The one idea — Non-determinism has layers: the roll, the seed — and even at temperature zero, GPU arithmetic itself can wobble across runs and hardware.
- How it is shown — Twin machines, identical settings, diverging at one spin; then T=0 twins diverging anyway via floating-point order.
- The trap to avoid — "Inconsistent = broken" — variation is designed; true determinism is available but has costs, and API-side model updates are a different beast entirely.
- What it sets up — One roll ≠ the model's ability.
Turn the randomness all the way off and the same prompt can still give different answers. That last wobble hides in the GPU arithmetic itself.
The one idea
Non-determinism has layers: the roll, the seed — and even at temperature zero, GPU arithmetic itself can wobble across runs and hardware.
Ask the identical question twice — identical settings, identical everything — and get two different answers. It's dice, in three layers. Count them with me. Layer one you now own: the sampler rolls, by design. And divergence compounds — one different early word changes the context, which changes every subsequent market, which changes every subsequent roll. Two runs don't differ by one word; they fork into different documents. The dice are loaded the same; the paths are not. Layer two: the seed — the starting state of the random number generator feeding the wheel. Fix it, and the same rolls replay exactly; your randomness becomes reproducible, which is gold for debugging. Some APIs let you pin it.
How it works — the demo
Twin machines, identical settings, diverging at one spin; then T=0 twins diverging anyway via floating-point order.
And here's where the story should end. It doesn't. Layer three, the one that surprises engineers: even at temperature zero — no dice at all — runs can differ. Massively parallel chips add thousands of numbers in whatever order workers finish, and floating-point addition isn't perfectly order-stable: a-plus-b-plus-c can differ, microscopically, from c-plus-b-plus-a. Ninety-nine times in a hundred that vapor-thin difference changes nothing. But when two favorites stand nearly tied, it flips the winner — and the fork compounds from there. The dice go all the way down to the silicon. And one more layer that isn't dice at all: behind an API, the model itself can change — silent updates, new versions, different routing. If last month's prompt behaves differently today, you may be talking to a different machine. Different beast, same symptom; worth ruling out first.
The trap to avoid
"Inconsistent = broken" — variation is designed; true determinism is available but has costs, and API-side model updates are a different beast entirely.
Why it matters — and what’s next
One roll ≠ the model's ability.
The trap: filing variation as a defect. The wobble is mostly the cure from episode fifty-two, working. When you truly need repeatability — tests, audits, regulated flows — you can buy it: temperature zero, pinned seed, matched hardware and versions. Just know the bill: you've re-created the greedy machine, dull spots included. Choose per use-case, not per ideology. So: the roll, the seed, the silicon, and the moving target — every layer of "why did it answer differently?" named. Now watch what those single rolls become at scale. One word, fed back, becoming a paragraph — the loop this whole act has been circling. Next.
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:30 of narration
Ask the identical question twice — identical settings, identical everything — and get two different answers. It's dice, in three layers. Count them with me.
Layer one you now own: the sampler rolls, by design. And divergence compounds — one different early word changes the context, which changes every subsequent market, which changes every subsequent roll. Two runs don't differ by one word; they fork into different documents. The dice are loaded the same; the paths are not.
Layer two: the seed — the starting state of the random number generator feeding the wheel. Fix it, and the same rolls replay exactly; your randomness becomes reproducible, which is gold for debugging. Some APIs let you pin it. And here's where the story should end. It doesn't.
Layer three, the one that surprises engineers: even at temperature zero — no dice at all — runs can differ. Massively parallel chips add thousands of numbers in whatever order workers finish, and floating-point addition isn't perfectly order-stable: a-plus-b-plus-c can differ, microscopically, from c-plus-b-plus-a. Ninety-nine times in a hundred that vapor-thin difference changes nothing. But when two favorites stand nearly tied, it flips the winner — and the fork compounds from there. The dice go all the way down to the silicon.
And one more layer that isn't dice at all: behind an API, the model itself can change — silent updates, new versions, different routing. If last month's prompt behaves differently today, you may be talking to a different machine. Different beast, same symptom; worth ruling out first.
The trap: filing variation as a defect. The wobble is mostly the cure from episode fifty-two, working. When you truly need repeatability — tests, audits, regulated flows — you can buy it: temperature zero, pinned seed, matched hardware and versions. Just know the bill: you've re-created the greedy machine, dull spots included. Choose per use-case, not per ideology.
So: the roll, the seed, the silicon, and the moving target — every layer of "why did it answer differently?" named. Now watch what those single rolls become at scale. One word, fed back, becoming a paragraph — the loop this whole act has been circling. Next.