Did it memorize you?

Researchers have prompted AI models into reciting real training text word for word — matched character for character to source pages, personal information included. Here's when that happens.

The one idea

Weights sometimes compress and sometimes copy — memorization is real and measured; repetition and rarity decide which happens.

Two fates await everything the furnace reads. Most of it dissolves — patterns kept, wording gone. But some of it fossilizes: pressed into the weights nearly verbatim, recoverable. Sometimes it learned the pattern. Sometimes it kept your data. Tonight, the honest split. The default is compression, and arithmetic enforces it. The weight-file is thousands of times smaller than the corpus — episode three's oldest fact — so keeping everything verbatim is physically impossible. Regularities that appear across thousands of pages get kept; the individual wordings that carried them evaporate. That's generalization, and it's most of what training does. But two conditions defeat compression. Repetition: text the crawl carried a thousand times — famous passages, boilerplate, licenses — gets hammered into the weights verbatim by sheer re-exposure.

How it works — the demo

The two fates of a passage: dissolving into shared patterns vs. pressed in verbatim; researchers extracting real training text.

And weirdness: strings so unlike anything else that no shared pattern can absorb them — unique identifiers, odd name-and-number pairings — get filed as exceptions, wording intact. Repeated, or unrepeatable: that's what the machine keeps raw. This is measured, not speculated. Researchers have prompted models into continuing half-remembered prefixes and recovered real training text, verbatim — matched character for character to source pages. Including, in published attacks, personal information. Providers patch, attackers adapt — an arms race whose existence settles the question. Memorization is a measurable minority behavior. So — did it memorize you? The answer is a probability with drivers. How many copies of your text entered the funnel: one blog post is pattern-food; a string mirrored across a thousand pages is fossil-risk. How distinctive your strings are. And model size — bigger towers demonstrably memorize more.

The trap to avoid

Both absolutisms — "it's a copy of the internet" and "it can't memorize anything" — are wrong; the split is quantitative, and dedup is a privacy control.

Why it matters — and what’s next

Verbatim recall vs pattern recall.

Which reframes deduplication as privacy machinery: collapsing copies before training is the single most effective anti-memorization control the funnel has. The trap runs both directions, loudly, in public debate. "It's just a giant copy of the internet" — the arithmetic forbids it; a thousand-fold compression can't be a copy. "Models can't memorize, it's all statistics" — the extractions refute it, character for character. The truth is quantitative: overwhelmingly compression, with a real verbatim tail. Courtrooms and dinner tables would both argue better knowing the split. So the weights keep patterns, and sometimes fossils — and either way, the keeping is expensive. Every nudge of every weight, trillions of times, on thousands of machines, for months. Hanging over this whole hall is a meter spinning too fast to read. Next episode we read it. The price tag.

This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.