The knowledge hub
Every episode, supplement and Frontier Dispatch as a readable article — with the cinematic render, the full transcript, and the one idea that matters. 276 articles and counting.
02
The prediction engine
37 articles#26Ep2:30
A neuron is just multiply-and-add.
The atomic unit: inputs times weights, summed, squashed — mechanically, a similarity check between the input and a stored stencil.
#27Ep2:30
What a 'weight' actually is.
A weight is one tunable number — the parameters in the headlines are exactly these, everything learned lives in them, and no single one…
#28Ep2:30
Why parameter count isn't intelligence.
Parameter count is capacity, not smarts — data, training compute, architecture, and post-training decide what fills the capacity.
#29Ep2:15
A layer: many neurons at once.
A layer is many neurons reading the same input in parallel — a bank of questions whose answers form the next vector; a matrix multiply is…
#30Ep2:45
The model's working memory: the residual stream.
Each token carries one running vector through the whole machine; every component reads from it and ADDS its contribution back…
#31Ep2:45
The problem attention solves.
A word's meaning depends on other words — the machine needs learned, content-based, any-to-any communication between token streams.
#32Ep3:00
Attention, the library metaphor.
Query, Key, Value: every token files a request slip, advertises a spine label, and offers contents — match slips to labels, blend the books.
#33Ep2:30
How the model scores a 'match.'
Relevance is a dot product between query and key — the same instrument as word similarity and the neuron — scaled by √d to keep the race…
#34Ep2:30
Turning scores into focus (softmax).
Softmax converts raw scores into a budget summing to 1 — exponential sharpening of winners, nothing ever exactly zero.
#35Ep2:30
The payoff: a weighted blend.
Attention's output is a weighted sum of value vectors, added to the token's stream — context physically installed, meaning physically moved.
#36Ep2:30
Doing it for every word at once.
Attention is one big matrix operation over all tokens in parallel — with a triangular mask forbidding each token from seeing its future.
#37Ep2:45
Why long prompts get expensive fast.
Attention is O(n²): double the tokens, quadruple the pair-work — the curve behind long-context pricing and a decade of engineering arms…
#38Ep2:30
Many heads, many kinds of looking.
Multi-head attention runs several independent attention patterns in parallel — separate lenses, separate budgets, separate blends…
#39Ep2:30
What attention heads secretly specialize in.
Cracked open, models reveal real specialist heads — previous-token heads, reference-tracking heads, and induction heads that notice "this…
#40Ep2:30
Where knowledge actually lives (the MLP).
The feed-forward block stores learned facts and patterns — attention routes information; the MLP is where it's kept.
#41Ep2:30
A transformer block, assembled.
Attention + MLP + residual adds (+ a volume-leveler before each) = one repeatable block — talk, then think.
#42Ep2:15
Stack the brick 100 times.
The whole model is one brick repeated — abstraction rises floor by floor, and there's no exotic machinery deeper in.
#43Ep2:15
Depth vs. width, and what each buys.
Deeper = more abstraction steps; wider = more capacity per step — and the bill scales differently: width is priced quadratically.
#44Ep2:30
The context window, explained.
The context window is the finite number of tokens the machine can attend to at once — a desk, not a memory; on the desk is "now," off the…
#45Ep2:30
Why it 'forgets' the start of a chat.
When a conversation outgrows the window, the app trims — the model never forgets what it never saw; and even in-window content isn't equal…
#46Ep3:00
The trick that makes chat affordable (KV cache).
Past tokens' keys and values never change — compute them once, shelve them, and each new token only pays for itself.
#47Ep2:30
Why the KV cache eats your memory.
The cache lives in VRAM and grows per token held — long context times many users can dwarf the model itself.
#48Ep2:45
Prefill vs. decode: two different machines.
Reading your prompt and writing the reply are physically different workloads — a parallel compute-bound flood, then a serial memory-bound…
#49Ep3:00
From the last layer to a word (the head).
The output head projects the final stream vector onto the whole vocabulary — one learned matrix, one score per token, computed fresh every…
#50Ep2:45
Logits: the model's raw opinion.
Logits are unnormalized scores over every candidate — the richest object in the pipeline, and almost all of it is discarded before you see…
#51Ep2:30
Turning scores into probabilities.
Softmax over the logits yields the next-token probability distribution — same machine that focused attention, now pricing the vocabulary.
#52Ep2:45
Picking the next word (sampling).
The model samples from the distribution rather than always taking the top — because always-greedy text is measurably repetitive and dead.
#53Ep2:30
The temperature knob.
Temperature divides the logits before softmax — low sharpens toward the favorite, high flattens toward chaos; it redistributes probability…
#54Ep2:30
top-p, top-k, min-p — the other dials.
Truncation strategies decide which tokens are even eligible before the roll — three bouncers with three door policies.
#55Ep2:30
Why the same prompt gives different answers.
Non-determinism has layers: the roll, the seed — and even at temperature zero, GPU arithmetic itself can wobble across runs and hardware.
#56Ep2:45
How it writes a whole paragraph.
Autoregression: each generated token is appended and fed back to predict the next — a loop with no plan buffer anywhere.
#57Ep2:30
Why it can't 'take back' a word.
Left-to-right generation makes committed tokens permanent context — the machine builds on its own errors, confidently.
#58Ep2:15
When it decides to stop.
An end-of-sequence token — predicted like any word — plus server-side stop rules terminate generation; stopping is a prediction wearing a…
#59Ep3:00
One token's full journey.
The act finale: replay the entire pipeline end to end — one word riding the whole machine, every subsystem named, no step skipped.
U1Supp2:45
Five ways you're using AI wrong.
Five common mistakes — vague asks, no context, no examples, no role or goal, and treating one reply as final — all trace to a single…
U2Supp2:45
Trust, but verify: knowing when it's guessing.
AI has no "I don't know" reflex — it produces plausible text — so it's reliable when transforming material you gave it and risky when…
U3Supp2:45
Your AI workflow: from oracle to thinking partner.
The mindset shift from oracle to thinking partner: ground it in your documents, use it to argue and critique and question, iterate in…