hawkes.aiKey ideas
- The one idea — A weight is one tunable number — the parameters in the headlines are exactly these, everything learned lives in them, and no single one means anything alone.
- How it is shown — Zooming from the downloaded weights file → a matrix → one cell → its one tiny job in one multiply.
- The trap to avoid — Reading individual weights for meaning — knowledge lives in patterns across millions, like a face in no single pixel.
- What it sets up — Training changes ONLY these numbers.
Everything a model learned, from grammar to geography, exists as nothing but the final setting of billions of stored numbers. Training changes only them. The architecture never moves.
The one idea
A weight is one tunable number — the parameters in the headlines are exactly these, everything learned lives in them, and no single one means anything alone.
When people say a model has seven hundred billion parameters — these are them. Let's hold one up to the light. A weight is one stored number with one job: it sits inside a neuron and multiplies one input. That's the whole biography. "Parameter" is the formal word for the same thing — a number the training process is allowed to tune. Where do they live? In matrices — big rectangular grids of numbers. A model is stacks of these grids, floor after floor. You've already met one: the embedding table, one row per token, was your first weight matrix. Every stencil in every neuron is a row in another.
How it works — the demo
Zooming from the downloaded weights file → a matrix → one cell → its one tiny job in one multiply.
Now the headline numbers decode themselves. Seven billion, seventy billion, seven hundred billion — literal counts of these stored numbers. And that file you downloaded in episode three? It is exactly these numbers, written to disk, nothing else. The whole mind, enumerable. Here's the sentence that organizes the next act: training changes only these numbers. The architecture — the arrangement of neurons and layers — is fixed before training and identical after. Everything the model learned, from grammar to geography to your favorite poem's rhythm, exists as nothing but the final configuration of these grids. The trap: reading a single weight for meaning. Weight four billion and twelve is not "the Paris weight." Zoom in and every number is anonymous; zoom out and knowledge appears as patterns across millions of them — the way a face lives in no single pixel of a photograph.
The trap to avoid
Reading individual weights for meaning — knowledge lives in patterns across millions, like a face in no single pixel.
Why it matters — and what’s next
Training changes ONLY these numbers.
This is why "just look inside the model" turns out to be a research field, not a weekend task. Hold this framing and half the industry's vocabulary unlocks for free. Copying a model is copying numbers. Quantization — coming in the hardware act — is storing the same numbers more cheaply. Fine-tuning is adjusting a few of them. It's numbers all the way down. So — the obvious next question. If everything lives in the count of numbers… does a bigger count mean a smarter model? Next episode: the counterexample the marketing hopes you never check.
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:30 of narration
When people say a model has seven hundred billion parameters — these are them. Let's hold one up to the light.
A weight is one stored number with one job: it sits inside a neuron and multiplies one input. That's the whole biography. "Parameter" is the formal word for the same thing — a number the training process is allowed to tune.
Where do they live? In matrices — big rectangular grids of numbers. A model is stacks of these grids, floor after floor. You've already met one: the embedding table, one row per token, was your first weight matrix. Every stencil in every neuron is a row in another.
Now the headline numbers decode themselves. Seven billion, seventy billion, seven hundred billion — literal counts of these stored numbers. And that file you downloaded in episode three? It is exactly these numbers, written to disk, nothing else. The whole mind, enumerable.
Here's the sentence that organizes the next act: training changes only these numbers. The architecture — the arrangement of neurons and layers — is fixed before training and identical after. Everything the model learned, from grammar to geography to your favorite poem's rhythm, exists as nothing but the final configuration of these grids.
The trap: reading a single weight for meaning. Weight four billion and twelve is not "the Paris weight." Zoom in and every number is anonymous; zoom out and knowledge appears as patterns across millions of them — the way a face lives in no single pixel of a photograph. This is why "just look inside the model" turns out to be a research field, not a weekend task.
Hold this framing and half the industry's vocabulary unlocks for free. Copying a model is copying numbers. Quantization — coming in the hardware act — is storing the same numbers more cheaply. Fine-tuning is adjusting a few of them. It's numbers all the way down.
So — the obvious next question. If everything lives in the count of numbers… does a bigger count mean a smarter model? Next episode: the counterexample the marketing hopes you never check.