hawkes.aiKey ideas
- The one idea — Calibration is whether a model's confidence matches its accuracy — 70% confident should mean right 70% of the time. Models carry a real internal uncertainty signal (in the token probabilities), but their expressed confidence often doesn't track it, so "sounding sure" ≠ "being right."
- How it is shown — A model's token probabilities being fairly calibrated on facts, while its worded confidence ("I'm certain!") stays high even when wrong — the internal signal and the outward tone diverging.
- The trap to avoid — Reading the model's fluent, confident tone as real confidence — expressed confidence is a learned style, not a truth gauge; the useful signal is in the probabilities.
- What it sets up — There IS an internal uncertainty signal — in the logits. can we read it out and use it?
Deep inside, the model carries a real signal of how unsure it is — in the token probabilities. The confident tone you hear? A learned style that often ignores it.
The one idea
Calibration is whether a model's confidence matches its accuracy — 70% confident should mean right 70% of the time. Models carry a real internal uncertainty signal (in the token probabilities), but their expressed confidence often doesn't track it, so "sounding sure" ≠ "being right."
If the model is always confident on the surface, here's the real question: does it know, inside, when it's on shaky ground? The technical name for this is calibration, and the answer turns out to be a fascinating "partly." Calibration means your confidence matches your accuracy. A perfectly calibrated model that says it's seventy percent sure is right about seventy percent of the time; ninety-nine percent means almost always right. It isn't about being smart — it's about your confidence being honest. And there is a real signal inside. Recall the logits — the probabilities over the next token. When the model is genuinely unsure, that distribution is often flatter, spread across many options; when it's sure, it's sharp and peaked. Remarkably, base models are often well-calibrated in those raw probabilities. But here's the catch: the confidence you hear is not the confidence inside.
How it works — the demo
A model's token probabilities being fairly calibrated on facts, while its worded confidence ("I'm certain!") stays high even when wrong — the internal signal and the outward tone diverging.
That fluent "I'm certain" tone is a learned writing style, not a readout of the probabilities. And training a model to be agreeable can make it worse-calibrated — more confident-sounding, especially when wrong. Tone and truth come apart. This is exactly how a model can be so fluently, convincingly wrong. The part that "knows" it's unsure — the flatter distribution — is buried under a surface that always sounds assured. Listen only to the words and you get no warning. The warning is in the numbers, not the prose. So the trap: taking the model's confident tone as real confidence. It isn't a truth gauge; it's a learned style.
The trap to avoid
Reading the model's fluent, confident tone as real confidence — expressed confidence is a learned style, not a truth gauge; the useful signal is in the probabilities.
Why it matters — and what’s next
There IS an internal uncertainty signal — in the logits. can we read it out and use it?
"I'm absolutely sure" tells you how the model writes, not how likely it is to be right. To gauge its uncertainty, don't read the prose — read the probabilities. So: does the model know when it's unsure? Partly. There's a genuine uncertainty signal in its token probabilities, even when its words don't show it. That's the opening we need. If the signal is in the logits, can we read it out and act on it? Next group: reading confidence from the logits.
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:45 of narration
If the model is always confident on the surface, here's the real question: does it know, inside, when it's on shaky ground? The technical name for this is calibration, and the answer turns out to be a fascinating "partly."
Calibration means your confidence matches your accuracy. A perfectly calibrated model that says it's seventy percent sure is right about seventy percent of the time; ninety-nine percent means almost always right. It isn't about being smart — it's about your confidence being honest.
And there is a real signal inside. Recall the logits — the probabilities over the next token. When the model is genuinely unsure, that distribution is often flatter, spread across many options; when it's sure, it's sharp and peaked. Remarkably, base models are often well-calibrated in those raw probabilities.
But here's the catch: the confidence you hear is not the confidence inside. That fluent "I'm certain" tone is a learned writing style, not a readout of the probabilities. And training a model to be agreeable can make it worse-calibrated — more confident-sounding, especially when wrong. Tone and truth come apart.
This is exactly how a model can be so fluently, convincingly wrong. The part that "knows" it's unsure — the flatter distribution — is buried under a surface that always sounds assured. Listen only to the words and you get no warning. The warning is in the numbers, not the prose.
So the trap: taking the model's confident tone as real confidence. It isn't a truth gauge; it's a learned style. "I'm absolutely sure" tells you how the model writes, not how likely it is to be right. To gauge its uncertainty, don't read the prose — read the probabilities.
So: does the model know when it's unsure? Partly. There's a genuine uncertainty signal in its token probabilities, even when its words don't show it. That's the opening we need. If the signal is in the logits, can we read it out and act on it? Next group: reading confidence from the logits.