hawkes.aiKey ideas
- The one idea — Video generation is diffusion extended across time — the hard part isn't any single frame, it's making hundreds of frames agree with each other so the whole thing reads as one coherent motion.
- How it is shown — A single perfect frame multiplying into a strip of frames that must all stay consistent; "time" added as a third axis to the carving, plus the compute cost of doing it hundreds of times.
- The trap to avoid — Judging a video model by one cherry-picked frame or a hand-picked two-second clip — the real test is coherence across the whole clip.
- What it sets up — The frames must agree — but they drift, morph, and flicker.
Your eye forgives a weird photo. It will not forgive a flicker. That's why video is generative AI's hardest problem — hundreds of frames, zero disagreements allowed.
The one idea
Video generation is diffusion extended across time — the hard part isn't any single frame, it's making hundreds of frames agree with each other so the whole thing reads as one coherent motion.
We've built a single AI image all act. Now ask for two seconds of video: not one picture but dozens of frames a second, each agreeing with the last. One good frame is easy. A thousand that don't drift or flicker is the hardest problem in generative AI. A still has two axes — width and height. Video adds a third: time. Now every frame must be two things: a good picture on its own, and consistent with its neighbors. Miss the first and one frame looks off. Miss the second and the whole clip falls apart — even when each frame alone is gorgeous. And it's brutally expensive.
How it works — the demo
A single perfect frame multiplying into a strip of frames that must all stay consistent; "time" added as a third axis to the carving, plus the compute cost of doing it hundreds of times.
A still is one image to carve; a few seconds of video is hundreds, plus the relationships between them. Even in the compressed latent space from episode one-fifty-four — the trick that made images cheap — there's far more to compute. That's why AI clips are short and cost real money. So how do models add time? The core move: let frames see each other. Video models add temporal attention — links letting frame ten look at nine and eleven and stay consistent. Some carve the whole clip as one block of width, height, and time. Either way, it denoises a coordinated sequence, not a lone picture. But temporal attention only softens the problem. Holding a face, a shirt, an object, and a scene's physics steady across dozens of frames is genuinely hard — and we're merciless judges, catching any flicker or morph.
The trap to avoid
Judging a video model by one cherry-picked frame or a hand-picked two-second clip — the real test is coherence across the whole clip.
Why it matters — and what’s next
The frames must agree — but they drift, morph, and flicker.
A still gets one shot at plausible; a video must be plausible dozens of times a second and never contradict itself. The trap: judging a video model by one stunning frame, or a hand-picked two-second clip. Any decent model makes a gorgeous frame — solved episodes ago. The real test is coherence over time: does the character stay one person, does motion obey physics, does it hold past a blink? Don't watch the highlight. Watch for the drift. So video generation is the fight to keep hundreds of frames agreeing across that third axis, time. Watch any AI clip closely and you'll see it lost in small ways: a shirt shifting color, a face becoming someone else. Those aren't random glitches — they're the specific ways a model forgets what it just drew. Next: why video models forget between frames.
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:45 of narration
We've built a single AI image all act. Now ask for two seconds of video: not one picture but dozens of frames a second, each agreeing with the last. One good frame is easy. A thousand that don't drift or flicker is the hardest problem in generative AI.
A still has two axes — width and height. Video adds a third: time. Now every frame must be two things: a good picture on its own, and consistent with its neighbors. Miss the first and one frame looks off. Miss the second and the whole clip falls apart — even when each frame alone is gorgeous.
And it's brutally expensive. A still is one image to carve; a few seconds of video is hundreds, plus the relationships between them. Even in the compressed latent space from episode one-fifty-four — the trick that made images cheap — there's far more to compute. That's why AI clips are short and cost real money.
So how do models add time? The core move: let frames see each other. Video models add temporal attention — links letting frame ten look at nine and eleven and stay consistent. Some carve the whole clip as one block of width, height, and time. Either way, it denoises a coordinated sequence, not a lone picture.
But temporal attention only softens the problem. Holding a face, a shirt, an object, and a scene's physics steady across dozens of frames is genuinely hard — and we're merciless judges, catching any flicker or morph. A still gets one shot at plausible; a video must be plausible dozens of times a second and never contradict itself.
The trap: judging a video model by one stunning frame, or a hand-picked two-second clip. Any decent model makes a gorgeous frame — solved episodes ago. The real test is coherence over time: does the character stay one person, does motion obey physics, does it hold past a blink? Don't watch the highlight. Watch for the drift.
So video generation is the fight to keep hundreds of frames agreeing across that third axis, time. Watch any AI clip closely and you'll see it lost in small ways: a shirt shifting color, a face becoming someone else. Those aren't random glitches — they're the specific ways a model forgets what it just drew. Next: why video models forget between frames.