hawkes.aiKey ideas
- The one idea — Cosine similarity — closeness is the angle between arrows, not ruler distance; direction carries the meaning.
- How it is shown — Two arrows from the origin, the angle shrinking; hot/cold hugging (the antonym surprise).
- The trap to avoid — High cosine = used-alike, not means-the-same — antonyms keep the same company; never read similarity as agreement.
- What it sets up — This exact operation is semantic search.
In the model's map of meaning, 'hot' and 'cold' are close neighbors. Opposites keep the same company — and that quirk bites real retrieval systems.
The one idea
Cosine similarity — closeness is the angle between arrows, not ruler distance; direction carries the meaning.
Two words are "close" when their arrows point the same way. Not when they're near. That distinction runs half the AI industry — here's why. The natural instinct: measure the map with a ruler — straight-line distance. In four thousand dimensions, that instinct betrays you. The workhorse of AI similarity is not a ruler. It's an angle. Here's the geometry — and then the trap that bites every retrieval system. Every embedding is an arrow from the origin. To compare two words: ignore how long their arrows are, compare where they point. Same direction — similarity near one.
How it works — the demo
Two arrows from the origin, the angle shrinking; hot/cold hugging (the antonym surprise).
Perpendicular — near zero. That measure is cosine similarity, and you have now met one of the most-computed numbers in applied AI. Why direction? Arrow length picks up how often training saw a word — frequency, emphasis. Direction is where the meaning settles. And in very high dimensions, ruler-distances bunch together — nearly everything is "far" — while angles stay informative. So the angle wins. The computation is one multiply-and-add pass — a dot product, normalized. Cheap enough to run billions of times. And that's precisely what "semantic search" means: your query's arrow, compared by angle against a million documents' arrows. Later, when we build systems that feed models documents, this exact operation decides what gets found.
The trap to avoid
High cosine = used-alike, not means-the-same — antonyms keep the same company; never read similarity as agreement.
Why it matters — and what’s next
This exact operation is semantic search.
Remember this little instrument; half the plumbing act runs on it. The trap: similar direction means used-alike, not means-the-same. "Hot" and "cold" live in nearly identical sentences — their arrows hug. Antonyms are neighbors, because opposites keep the same company. Never read a high cosine as agreement; read it as shared context. One question left on the map. Why four thousand coordinates — why not forty? What exactly do the extra dimensions buy? Last stop of the act. Next.
When AI search finds you a document, here's the actual test it ran: compare the angle between two arrows. Not the distance. The angle — billions of times.
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:30 of narration
Two words are "close" when their arrows point the same way. Not when they're near. That distinction runs half the AI industry — here's why.
The natural instinct: measure the map with a ruler — straight-line distance. In four thousand dimensions, that instinct betrays you. The workhorse of AI similarity is not a ruler. It's an angle. Here's the geometry — and then the trap that bites every retrieval system.
Every embedding is an arrow from the origin. To compare two words: ignore how long their arrows are, compare where they point. Same direction — similarity near one. Perpendicular — near zero. That measure is cosine similarity, and you have now met one of the most-computed numbers in applied AI.
Why direction? Arrow length picks up how often training saw a word — frequency, emphasis. Direction is where the meaning settles. And in very high dimensions, ruler-distances bunch together — nearly everything is "far" — while angles stay informative. So the angle wins.
The computation is one multiply-and-add pass — a dot product, normalized. Cheap enough to run billions of times. And that's precisely what "semantic search" means: your query's arrow, compared by angle against a million documents' arrows. Later, when we build systems that feed models documents, this exact operation decides what gets found. Remember this little instrument; half the plumbing act runs on it.
The trap: similar direction means used-alike, not means-the-same. "Hot" and "cold" live in nearly identical sentences — their arrows hug. Antonyms are neighbors, because opposites keep the same company. Never read a high cosine as agreement; read it as shared context.
One question left on the map. Why four thousand coordinates — why not forty? What exactly do the extra dimensions buy? Last stop of the act. Next.