hawkes.aiKey ideas
- The one idea — Facts are stored directionally. Train a model on "A is B" and it can answer A→B, but often can't answer B→A — the "reversal curse." Knowledge lives as one-way associations in the weights, not as symmetric database entries.
- How it is shown — A model that correctly answers "who is Tom Cruise's mother?" but fails "who is [that person]'s son?" — the same fact, one direction learned, the reverse not.
- The trap to avoid — Assuming the model has a symmetric, database-like store of facts — it has directional associations; knowing A→B doesn't guarantee B→A.
- What it sets up — If it stores facts so unlike a database, does it at least know when it's unsure?
A model can know Tom Cruise's mother is Mary Lee Pfeiffer, then draw a blank on who her son is. Same fact, backwards — gone. That's the reversal curse.
The one idea
Facts are stored directionally. Train a model on "A is B" and it can answer A→B, but often can't answer B→A — the "reversal curse." Knowledge lives as one-way associations in the weights, not as symmetric database entries.
Here's a strange, concrete symptom of how a model stores knowledge. It can know A is B, and completely fail to know B is A — the same fact, backwards. Researchers named it the reversal curse, and it says something deep about what model knowledge is. The classic example: a model taught "Tom Cruise's mother is Mary Lee Pfeiffer." Ask "who is Tom Cruise's mother?" and it answers correctly. Ask "who is Mary Lee Pfeiffer's son?" — the same fact — and it can draw a blank. One direction solid; the reverse gone. Why? Recall that knowledge lives in the weights, as learned associations. Training on "A is B" strengthens a pathway from A to B. It doesn't automatically build the reverse, from B back to A.
How it works — the demo
A model that correctly answers "who is Tom Cruise's mother?" but fails "who is [that person]'s son?" — the same fact, one direction learned, the reverse not.
The model learned to continue A with B, not B with A. The association has a direction. Here's the key. A database stores a fact once and queries it any direction — symmetric by design. A model stores a direction of prediction: its "facts" are strong next-token tendencies, and a tendency points one way. Same information, one way in. Not a lookup table — a bundle of leanings. This isn't a one-off bug — it's a window into model knowledge. What feels like knowing a fact is a bundle of directional associations, shaped by how the training text was written. Rephrase the same truth, and what the model can retrieve shifts.
The trap to avoid
Assuming the model has a symmetric, database-like store of facts — it has directional associations; knowing A→B doesn't guarantee B→A.
Why it matters — and what’s next
If it stores facts so unlike a database, does it at least know when it's unsure?
The trap: picturing the model as a clean, symmetric database you can query from any angle. It isn't. Knowing A to B doesn't guarantee B to A. When a model fails a backwards question, it isn't dumb — it's showing you its knowledge is one-way associations, not stored records. The gaps are the mechanism showing through. So facts are stored directionally: the reversal curse is what you get when a next-token predictor stands in for a database. Its knowledge is shaped — and mis-shaped — by the training text. Which raises the question underneath: does the model know when it doesn't know? Next: does it know when it's unsure?
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:30 of narration
Here's a strange, concrete symptom of how a model stores knowledge. It can know A is B, and completely fail to know B is A — the same fact, backwards. Researchers named it the reversal curse, and it says something deep about what model knowledge is.
The classic example: a model taught "Tom Cruise's mother is Mary Lee Pfeiffer." Ask "who is Tom Cruise's mother?" and it answers correctly. Ask "who is Mary Lee Pfeiffer's son?" — the same fact — and it can draw a blank. One direction solid; the reverse gone.
Why? Recall that knowledge lives in the weights, as learned associations. Training on "A is B" strengthens a pathway from A to B. It doesn't automatically build the reverse, from B back to A. The model learned to continue A with B, not B with A. The association has a direction.
Here's the key. A database stores a fact once and queries it any direction — symmetric by design. A model stores a direction of prediction: its "facts" are strong next-token tendencies, and a tendency points one way. Same information, one way in. Not a lookup table — a bundle of leanings.
This isn't a one-off bug — it's a window into model knowledge. What feels like knowing a fact is a bundle of directional associations, shaped by how the training text was written. Rephrase the same truth, and what the model can retrieve shifts.
The trap: picturing the model as a clean, symmetric database you can query from any angle. It isn't. Knowing A to B doesn't guarantee B to A. When a model fails a backwards question, it isn't dumb — it's showing you its knowledge is one-way associations, not stored records. The gaps are the mechanism showing through.
So facts are stored directionally: the reversal curse is what you get when a next-token predictor stands in for a database. Its knowledge is shaped — and mis-shaped — by the training text. Which raises the question underneath: does the model know when it doesn't know? Next: does it know when it's unsure?