The knowledge hub
Every episode, supplement and Frontier Dispatch as a readable article — with the cinematic render, the full transcript, and the one idea that matters. 276 articles and counting.
09
The limits
26 articles#213Ep2:45
Hallucination isn't a bug.
A model has no "I don't know" token — at every step it outputs a probability distribution over the next token and picks a plausible one…
#214Ep2:30
It knows A=B but not B=A.
Facts are stored directionally. Train a model on "A is B" and it can answer A→B, but often can't answer B→A — the "reversal curse."…
#215Ep2:45
Does the model know when it's unsure?
Calibration is whether a model's confidence matches its accuracy — 70% confident should mean right 70% of the time. Models carry a real…
#216Ep2:30
Reading confidence from the logits.
You can read a model's uncertainty straight from the logits: how much probability sits on the chosen token, summarized by entropy (peaked…
#217Ep2:45
Semantic entropy: catching confabulation.
Semantic entropy catches confabulation by sampling several answers and measuring whether they agree in meaning: if the model knows, they…
#218Ep3:00
Turning confidence into a GUARANTEE (conformal).
Conformal prediction converts a raw confidence score into a real guarantee: using a held-out calibration set, it picks a threshold so the…
#219Ep2:45
How the conformal gate forces 'I don't know.'
You wire the conformal bound into the harness as a gate: below the calibrated confidence threshold, the system is not allowed to answer…
#220Ep2:45
Reading the model's mind (interpretability).
Interpretability reads the model's internal state: by probing the residual stream — the vector carrying information through the layers…
#221Ep2:45
More ideas than neurons (superposition).
Models pack far more concepts than they have neurons by storing them in superposition — overlapping directions that share neurons — so a…
#222Ep2:45
It often knows the truth — and says otherwise.
Interpretability keeps finding the same unsettling thing: a model often internally represents a fact correctly while asserting the…
#223Ep2:45
Steering a model by editing its thoughts.
Because concepts are directions in the activations, you can steer a model by adding a concept vector to its residual stream at inference…
#224Ep2:30
Why steering is a nudge, not a knob.
Steering is a nudge, not a knob: the honesty direction is distributed, entangled with other features, and model-specific, so pushing it…
#225Ep2:45
The lethal trifecta.
The "lethal trifecta": give an AI agent private data + exposure to untrusted content + an outbound channel, and data exfiltration becomes…
#226Ep2:45
Jailbreaks attack a costume, not a rulebook.
A model's safety isn't a hard rulebook enforced by code — it's a trained disposition, a learned tendency to refuse, worn like a costume…
#227Ep2:45
Detect-and-block vs. structural constraint.
Two defense philosophies: detect-and-block (let the model act, then catch the bad output — reactive, probabilistic, an arms race) vs…
#228Ep2:45
Why 'a second AI checks the first' fails.
"Have a second AI check the first" is tempting but fails as a guarantee: the verifier shares the generator's blind spots — same data, same…
#229Ep2:45
Inside an eval harness.
A serious agentic eval isn't a quiz — it's a factory: task spec → sandboxed container → agent rollout → grader. Evals are mostly…
#230Ep2:30
pass@k, variance, and the judge problem.
One run proves nothing. Generation is stochastic, so a fair score needs pass@k (can it do it in k tries?), seeds and repeats (how…
#231Ep2:45
A grammar for actions, not just words.
The same constrained-decoding trick that forces valid JSON can whitelist the exact shape of legal actions — a bounding grammar so an…
#232Ep2:45
'Structurally impossible' — the honest version.
When a structural constraint makes something "impossible," the honest claim is about actions that can't execute, not thoughts the model…
#233Ep2:30
The deterministic checker (not another AI).
The thing enforcing the action boundary must be a fixed, deterministic parser — not a model. Its dumbness is the source of its trust…
#234Ep2:30
The industrial roots: validating protocol at the wire.
Validating a strict message format against a grammar, with a deterministic parser at the boundary that drops anything malformed, is an…
W1Supp2:45
Will it take your job? The honest version.
Economists distinguish tasks from jobs — AI automates tasks, and a job is a bundle of tasks, so the realistic near-term picture is…
W2Supp2:45
The regulation map.
The world is governing AI along different philosophies — the EU's comprehensive, risk-tiered AI Act versus the more sectoral, executive US…
W3Supp2:45
The AGI debate: why smart people disagree.
"AGI" has no agreed definition, and the disagreement about timelines is real and held by serious people — driven by different definitions…
W4Supp2:45
The honest ledger.
An honest reckoning puts real costs and real benefits on the same ledger — footprint, concentration, hidden labor, bias, and…