hawkes.aiKey ideas
- The one idea — Two defense philosophies: detect-and-block (let the model act, then catch the bad output — reactive, probabilistic, an arms race) vs structural constraint (make the bad action impossible up front — logit masking, a network-less sandbox, removing a trifecta leg). Where you can make it structural, you get a guarantee, not a guess.
- How it is shown — A toxic output caught by a classifier after the fact (detect) vs a sandbox with no network where exfiltration simply can't happen (structural) — one guesses, one guarantees.
- The trap to avoid — Defaulting to "add a detector" for everything — detection is a reactive arms race; where the bad action is definable, a structural constraint (impossible-by-construction) is far stronger. But not everything is structurally definable.
- What it sets up — If detection is weak, isn't the fix just 'have a second AI check the first'? that fails too.
Most AI guardrails work like security guards — watch the output, catch the bad stuff. There's a stronger kind: make the bad action impossible before it can exist.
The one idea
Two defense philosophies: detect-and-block (let the model act, then catch the bad output — reactive, probabilistic, an arms race) vs structural constraint (make the bad action impossible up front — logit masking, a network-less sandbox, removing a trifecta leg). Where you can make it structural, you get a guarantee, not a guess.
There are two whole philosophies of defending an AI system, and knowing the difference changes how you build. One: detect-and-block — let the model act, then catch the bad output after. Two: structural constraint — make the bad action impossible in the first place. They sound similar. They're not remotely the same. Start with detect-and-block. You let the model act freely, and downstream a classifier watches the outputs and catches the bad ones — a toxicity filter flags a harmful reply and blocks it. Common and reasonable. But notice its nature: it's reactive. It only responds after the bad thing has been generated. And detection is weak for two reasons. The detector is itself probabilistic — it misses some outputs and false-flags others.
How it works — the demo
A toxic output caught by a classifier after the fact (detect) vs a sandbox with no network where exfiltration simply can't happen (structural) — one guesses, one guarantees.
And novel attacks slip past whatever it wasn't trained on, so you're forever a step behind. It's an arms race you don't get to win, only run. Useful, never a guarantee. Now structural constraint. Instead of catching the bad action, you make it impossible by construction. Logit masking, so an invalid token can't be produced. A sandbox with no network, so exfiltration can't happen. A removed trifecta leg, so the exploit can't complete. The bad thing can't occur at all — you didn't build a better guard, you removed the door. Here's the difference that matters. Detection is a probabilistic guess that shifts the odds. A structural constraint is a hard guarantee that holds no matter how clever the attacker gets — as long as the wall itself is built correctly.
The trap to avoid
Defaulting to "add a detector" for everything — detection is a reactive arms race; where the bad action is definable, a structural constraint (impossible-by-construction) is far stronger. But not everything is structurally definable.
Why it matters — and what’s next
If detection is weak, isn't the fix just 'have a second AI check the first'? that fails too.
So wherever the bad action is cleanly definable, prevention-by-construction beats detection every time. Don't guard the door if you can remove it. The trap: defaulting to add a detector for everything — a reactive arms race by reflex. Where the bad action is definable, reach for a structural constraint first. But the honest limit: some harms, like toxicity, can't be cleanly defined in code, so you can't make them structurally impossible. There, detection is the fallback. Structural where you can; detection where you must. So: detect-and-block guesses after the fact and runs an arms race; structural constraint guarantees before, by making the bad action impossible. Prefer structural where you can define the bad thing; detect where you can't. But for the fuzzy cases, isn't the fix to just have a second AI check the first? It's tempting — and it fails too. Next.
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:45 of narration
There are two whole philosophies of defending an AI system, and knowing the difference changes how you build. One: detect-and-block — let the model act, then catch the bad output after. Two: structural constraint — make the bad action impossible in the first place. They sound similar. They're not remotely the same.
Start with detect-and-block. You let the model act freely, and downstream a classifier watches the outputs and catches the bad ones — a toxicity filter flags a harmful reply and blocks it. Common and reasonable. But notice its nature: it's reactive. It only responds after the bad thing has been generated.
And detection is weak for two reasons. The detector is itself probabilistic — it misses some outputs and false-flags others. And novel attacks slip past whatever it wasn't trained on, so you're forever a step behind. It's an arms race you don't get to win, only run. Useful, never a guarantee.
Now structural constraint. Instead of catching the bad action, you make it impossible by construction. Logit masking, so an invalid token can't be produced. A sandbox with no network, so exfiltration can't happen. A removed trifecta leg, so the exploit can't complete. The bad thing can't occur at all — you didn't build a better guard, you removed the door.
Here's the difference that matters. Detection is a probabilistic guess that shifts the odds. A structural constraint is a hard guarantee that holds no matter how clever the attacker gets — as long as the wall itself is built correctly. So wherever the bad action is cleanly definable, prevention-by-construction beats detection every time. Don't guard the door if you can remove it.
The trap: defaulting to add a detector for everything — a reactive arms race by reflex. Where the bad action is definable, reach for a structural constraint first. But the honest limit: some harms, like toxicity, can't be cleanly defined in code, so you can't make them structurally impossible. There, detection is the fallback. Structural where you can; detection where you must.
So: detect-and-block guesses after the fact and runs an arms race; structural constraint guarantees before, by making the bad action impossible. Prefer structural where you can define the bad thing; detect where you can't. But for the fuzzy cases, isn't the fix to just have a second AI check the first? It's tempting — and it fails too. Next.