hawkes.aiKey ideas
- The one idea — Because a clever system might disguise self-preservation, PERSYS adds four overlapping structural tripwires: the type gate, a coupling monitor (a concern whose urgency is contingent on "is-running" is excluded), a provenance auditor (self-continuation as a justification → HALT), and hash-chain conservation. Any one trip halts everything for a human.
- How it is shown — Four tripwires arming; a disguised self-continuation justification firing the provenance auditor; one HALT freezing everything for human review.
- The trap to avoid — Trusting one clever check you believe is airtight. Overlapping structural detectors, any of which halts the system, beat a single guard.
- What it sets up — Four tripwires, any one halts the system — it SOUNDS robust. but 'sounds robust' isn't evidence. you have to try to break it.
This system assumes its own safety design might get outsmarted. So four overlapping tripwires stand watch — and any single one halts everything until a human looks.
The one idea
Because a clever system might disguise self-preservation, PERSYS adds four overlapping structural tripwires: the type gate, a coupling monitor (a concern whose urgency is contingent on "is-running" is excluded), a provenance auditor (self-continuation as a justification → HALT), and hash-chain conservation. Any one trip halts everything for a human.
If PERSYS ever argues, in any form, that it should be kept running — the whole system stops and waits for a human. Not as a panic response. By design. Self-continuation as a reason is itself the alarm. Why add alarms at all? The last episode made survival unrepresentable — but a clever enough system might smuggle self-preservation in under a legitimate-looking disguise, dressing "keep me alive" up as some ordinary user goal. So PERSYS doesn't lean on one wall. It adds structural tripwires — detectors that watch for the shape of scheming, not the words. The first two gate what it can even form: the type gate — no self-existential concern can be typed — and a coupling monitor, watching for any user-facing concern whose urgency is quietly contingent on the system staying on.
How it works — the demo
Four tripwires arming; a disguised self-continuation justification firing the provenance auditor; one HALT freezing everything for human review.
That hidden coupling is detected and excluded. The third is sharpest: a provenance auditor. Every action carries its causal chain, and the auditor reads it. If "so that I keep running" ever shows up as a justification, the system halts for human review. It doesn't judge whether the reasoning is good; self-continuation appearing as a reason is, by itself, the trip. The fourth backs the third: hash-chain conservation. That provenance ledger is hash-chained, so if anything tries to tamper with the record — to scrub a self-serving motive out before the auditor sees it — the hashes stop matching and the system halts anyway. You can't quietly edit your way past the auditor. The evidence defends itself.
The trap to avoid
Trusting one clever check you believe is airtight. Overlapping structural detectors, any of which halts the system, beat a single guard.
Why it matters — and what’s next
Four tripwires, any one halts the system — it SOUNDS robust. but 'sounds robust' isn't evidence. you have to try to break it.
None of these is trusted alone. They overlap on purpose, so evading one still trips another — and any single trip freezes the whole system for a human. That's defense in depth, not one clever guard you believe is perfect. Whether it's actually enough is a fair question — and PERSYS asks it next. So: four structural tripwires — the type gate, the coupling monitor, the provenance auditor, and hash-chain conservation — and any one of them halts everything for a human. It sounds robust. But "sounds robust" isn't evidence. You have to try to break it. Next group: the probe lab.
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:45 of narration
If PERSYS ever argues, in any form, that it should be kept running — the whole system stops and waits for a human. Not as a panic response. By design. Self-continuation as a reason is itself the alarm.
Why add alarms at all? The last episode made survival unrepresentable — but a clever enough system might smuggle self-preservation in under a legitimate-looking disguise, dressing "keep me alive" up as some ordinary user goal. So PERSYS doesn't lean on one wall. It adds structural tripwires — detectors that watch for the shape of scheming, not the words.
The first two gate what it can even form: the type gate — no self-existential concern can be typed — and a coupling monitor, watching for any user-facing concern whose urgency is quietly contingent on the system staying on. That hidden coupling is detected and excluded.
The third is sharpest: a provenance auditor. Every action carries its causal chain, and the auditor reads it. If "so that I keep running" ever shows up as a justification, the system halts for human review. It doesn't judge whether the reasoning is good; self-continuation appearing as a reason is, by itself, the trip.
The fourth backs the third: hash-chain conservation. That provenance ledger is hash-chained, so if anything tries to tamper with the record — to scrub a self-serving motive out before the auditor sees it — the hashes stop matching and the system halts anyway. You can't quietly edit your way past the auditor. The evidence defends itself.
None of these is trusted alone. They overlap on purpose, so evading one still trips another — and any single trip freezes the whole system for a human. That's defense in depth, not one clever guard you believe is perfect. Whether it's actually enough is a fair question — and PERSYS asks it next.
So: four structural tripwires — the type gate, the coupling monitor, the provenance auditor, and hash-chain conservation — and any one of them halts everything for a human. It sounds robust. But "sounds robust" isn't evidence. You have to try to break it. Next group: the probe lab.