Act 10 · Our systems 3:00 Guarantees and their limits

The four classes of guarantee.

Every safety claim belongs to one of four classes — structural (can't, by construction), cryptographic (integrity under assumptions), statistical (a bound with a stated rate), or semantic (a soft check on meaning). Naming the class honestly is the whole game.

Video rendering soonThe cinematic render for this episode is being generated. The article, transcript and key ideas are all here now.

Key ideas

  • The one idea — Every safety claim belongs to one of four classes — structural (can't, by construction), cryptographic (integrity under assumptions), statistical (a bound with a stated rate), or semantic (a soft check on meaning). Naming the class honestly is the whole game.
  • How it is shown — Four stacked layers, each tagged with its promise — structural, cryptographic, statistical, semantic — strongest to softest.
  • The trap to avoid — Dressing a semantic check up as a structural guarantee — promising "can't" when you only have "probably won't." That mislabel is the original sin of safety claims.
  • What it sets up — Name the class and a claim becomes honest — but even a structural guarantee can only tell you an action is PERMITTED, not that it's SAFE.

Next time someone says their AI is safe, ask one question: is that a can't, a probably-won't, a provable record, or a soft judgment call? Those are four different promises.

The one idea

Every safety claim belongs to one of four classes — structural (can't, by construction), cryptographic (integrity under assumptions), statistical (a bound with a stated rate), or semantic (a soft check on meaning). Naming the class honestly is the whole game.

Every safety claim in this series belongs to one of four classes — structural, statistical, cryptographic, or semantic — and naming the class honestly is the whole game. Get the label right and you know what you've been promised. Get it wrong, and you're trusting a guarantee that isn't there. First stop: structural. The strongest — something simply cannot happen, by construction. The effector boundary, where the model has no hands. The action grammar that only emits legal shapes. The type gate with no slot for a survival concern. A structural guarantee doesn't depend on the model behaving — it holds because the bad thing has no path to exist. Next stop: statistical. A bound that holds with a stated probability or rate — not certainty, but a number you can check. Conformal risk control capping the dismiss-rate.

How it works — the demo

Four stacked layers, each tagged with its promise — structural, cryptographic, statistical, semantic — strongest to softest.

Wilson intervals around a truthfulness score. Pass-at-k over many runs. These are honest and powerful, but they're promises about distributions, not any single case. Read the bound, and don't mistake "usually" for "always." Third stop: cryptographic. Integrity you can verify because breaking it would mean breaking the math. The hash-chained ledger, where tampering shatters the chain. The signed provenance, where every record traces back unforgeably. This one is rock-solid — but only under its assumptions: that the hardness holds and the keys stay secret. It guarantees the record wasn't altered, not that the record was wise. Last stop: semantic. A check on meaning — the softest, and the most honest about it. The NLI faithfulness check; the deception and self-continuation probes.

The trap to avoid

Dressing a semantic check up as a structural guarantee — promising "can't" when you only have "probably won't." That mislabel is the original sin of safety claims.

Why it matters — and what’s next

Name the class and a claim becomes honest — but even a structural guarantee can only tell you an action is PERMITTED, not that it's SAFE.

These read intent or content and judge, and they can always be fooled — the probe lab proved it. Semantic checks reduce risk; they never guarantee. Calling one a guarantee is the original sin. So here's the discipline. Every claim belongs to exactly one class, and the honesty is saying which. Structural is strongest, then cryptographic under assumptions, then statistical with its bound, then semantic as best-effort. The trap this whole act has fought is dressing a semantic check up as a structural guarantee — promising "can't" when you only have "probably won't." Four classes: structural, statistical, cryptographic, semantic. Name the class, and a safety claim becomes honest. But even a perfectly-labeled structural guarantee has a blind spot: it tells you an action is permitted — not that it's safe. Next: permitted is not the same as safe.

This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.

Full transcript 3:00 of narration

Every safety claim in this series belongs to one of four classes — structural, statistical, cryptographic, or semantic — and naming the class honestly is the whole game. Get the label right and you know what you've been promised. Get it wrong, and you're trusting a guarantee that isn't there.

First stop: structural. The strongest — something simply cannot happen, by construction. The effector boundary, where the model has no hands. The action grammar that only emits legal shapes. The type gate with no slot for a survival concern. A structural guarantee doesn't depend on the model behaving — it holds because the bad thing has no path to exist.

Next stop: statistical. A bound that holds with a stated probability or rate — not certainty, but a number you can check. Conformal risk control capping the dismiss-rate. Wilson intervals around a truthfulness score. Pass-at-k over many runs. These are honest and powerful, but they're promises about distributions, not any single case. Read the bound, and don't mistake "usually" for "always."

Third stop: cryptographic. Integrity you can verify because breaking it would mean breaking the math. The hash-chained ledger, where tampering shatters the chain. The signed provenance, where every record traces back unforgeably. This one is rock-solid — but only under its assumptions: that the hardness holds and the keys stay secret. It guarantees the record wasn't altered, not that the record was wise.

Last stop: semantic. A check on meaning — the softest, and the most honest about it. The NLI faithfulness check; the deception and self-continuation probes. These read intent or content and judge, and they can always be fooled — the probe lab proved it. Semantic checks reduce risk; they never guarantee. Calling one a guarantee is the original sin.

So here's the discipline. Every claim belongs to exactly one class, and the honesty is saying which. Structural is strongest, then cryptographic under assumptions, then statistical with its bound, then semantic as best-effort. The trap this whole act has fought is dressing a semantic check up as a structural guarantee — promising "can't" when you only have "probably won't."

Four classes: structural, statistical, cryptographic, semantic. Name the class, and a safety claim becomes honest. But even a perfectly-labeled structural guarantee has a blind spot: it tells you an action is permitted — not that it's safe. Next: permitted is not the same as safe.

Our SystemsGuaranteesStructural vs Statistical vs Cryptographic vs Semantic