hawkes.aiKey ideas
- The one idea — The most economically deployed AI on Earth isn't a chatbot — it's ranking and recommendation: collaborative filtering to learned embeddings to deep ranking at planetary scale, the quiet engines that shaped attention and commerce for a decade.
- How it is shown — A person-by-item grid with its blanks filled; users and items becoming points in a shared embedding space; a ranking funnel from millions to a handful in milliseconds; the objective-beacon and its shadow.
- The trap to avoid — Treating the ranker as neutral — it relentlessly optimizes whatever objective you hand it, and "engagement" carries real trade-offs.
Every feed you scroll. Every "recommended for you." Every ad that appears in the gap.
The one idea
The most economically deployed AI on Earth isn't a chatbot — it's ranking and recommendation: collaborative filtering to learned embeddings to deep ranking at planetary scale, the quiet engines that shaped attention and commerce for a decade.
Behind all of it sits the same quiet kind of AI — and by money and reach, it is the most deployed AI ever built. It shaped a decade of attention before chatbots existed. It began with a simple idea: collaborative filtering. Line up every person against every item they rated, and most of that grid is blank. The blanks are predictable — people who agreed before tend to agree again. Fill them by that logic and you can recommend something never explicitly entered. Then the representation got smarter. Every user and every item becomes a vector — a point in a shared space, learned from behavior. Close together means similar taste. If that word, embeddings, sounds familiar, it should: it's the same idea this course teaches for search and retrieval.
How it works — the demo
A person-by-item grid with its blanks filled; users and items becoming points in a shared embedding space; a ranking funnel from millions to a handful in milliseconds; the objective-beacon and its shadow.
The recommender got there first. At scale, this becomes a funnel. From millions of candidate items, a fast first pass keeps a few thousand, then a heavy ranking model scores each for this person, this moment — in milliseconds, billions of times a day. For years, this quietly consumed much of the world's inference compute. But a ranker only chases the goal you give it. Give it clicks, watch-time, engagement — and it will find, relentlessly, whatever holds attention. Sometimes that's what genuinely helps you. Sometimes it's outrage, clickbait, and the rabbit hole. The math is neutral; the objective is a choice, and that choice shapes what billions see. The scale is staggering.
The trap to avoid
Treating the ranker as neutral — it relentlessly optimizes whatever objective you hand it, and "engagement" carries real trade-offs.
Why it matters — and what’s next
Feeds, search ranking, ads, shopping, streaming — this one family of models runs the attention economy and much of online commerce. It was the most consequential AI in your life for a decade. You never saw it, because it never had a face. So hold the thread. The embeddings that pick your next video are cousins of the embeddings this course uses to fetch the right document. Same idea, different job. Two branches down — one silent in the lab, one silent in your pocket. The next one is loud, clumsy, and harder than it looks: robots.
This is a supplement in AI: Zero → Frontier — a side-trip that deepens the act it sits beside, one file and one loop at a time.
Full transcript 2:45 of narration
Every feed you scroll. Every "recommended for you." Every ad that appears in the gap. Behind all of it sits the same quiet kind of AI — and by money and reach, it is the most deployed AI ever built.
It shaped a decade of attention before chatbots existed. It began with a simple idea: collaborative filtering. Line up every person against every item they rated, and most of that grid is blank.
The blanks are predictable — people who agreed before tend to agree again. Fill them by that logic and you can recommend something never explicitly entered. Then the representation got smarter.
Every user and every item becomes a vector — a point in a shared space, learned from behavior. Close together means similar taste. If that word, embeddings, sounds familiar, it should: it's the same idea this course teaches for search and retrieval.
The recommender got there first. At scale, this becomes a funnel. From millions of candidate items, a fast first pass keeps a few thousand, then a heavy ranking model scores each for this person, this moment — in milliseconds, billions of times a day.
For years, this quietly consumed much of the world's inference compute. But a ranker only chases the goal you give it. Give it clicks, watch-time, engagement — and it will find, relentlessly, whatever holds attention.
Sometimes that's what genuinely helps you. Sometimes it's outrage, clickbait, and the rabbit hole. The math is neutral; the objective is a choice, and that choice shapes what billions see.
The scale is staggering. Feeds, search ranking, ads, shopping, streaming — this one family of models runs the attention economy and much of online commerce. It was the most consequential AI in your life for a decade.
You never saw it, because it never had a face. So hold the thread. The embeddings that pick your next video are cousins of the embeddings this course uses to fetch the right document.
Same idea, different job. Two branches down — one silent in the lab, one silent in your pocket. The next one is loud, clumsy, and harder than it looks: robots.