AI: Zero Frontier

The knowledge hub

Every episode, supplement and Frontier Dispatch as a readable article — with the cinematic render, the full transcript, and the one idea that matters. 276 articles and counting.

FD

Frontier Dispatch

1 article

China just shipped the world's biggest open model. You already understand why.

A record-breaking model launch is only shocking if you don't know the shape of the field — and you do.

00

Orientation

7 articles

You typed a sentence. Here's where it went.

A round-trip tour of what happens between your message and the reply — no jargon.

It's a guesser, not a search engine.

The model predicts the next chunk of text; it does not look answers up.

Download a brain. Unplug the internet.

A model is one weights file plus a tiny loop of arithmetic — no database, no internet inside.

One idea explains hallucination AND creativity.

Generation is sampling from a distribution — invention welcomed is "creativity," invention where facts were required is "hallucination."…

The whole machine, on one screen.

The full pipeline on one map — tokenizer → embeddings → stacked blocks (attention + MLP) → head → logits → sampler → loop — and this…

Why smart people disagree about AI.

Every AI hot take describes a real layer of the stack — the error is totalizing one layer into the whole verdict.

The only promise this series makes.

The contract: no unexplained words, one idea per episode, failures on the syllabus, simplifications flagged — in exchange for watching in…

01

Text becomes numbers

18 articles

Computers can't see letters.

Text is numbers all the way down — bytes, encodings, glyphs are stacked agreements, and the letter exists only at the final rendering step.

Why 'é' is a trap.

Unicode gives é two legal spellings — identical glyphs, different bytes — so "looks identical" means nothing below the glyph layer…

Models don't read words. They read tokens.

The token — a learned chunk from a fixed menu — is the atomic unit of AI: of pricing, context, speed, and failure.

Why 'strawberry' breaks the AI.

Tokenization is why models miscount letters — they can't see inside a token.

Why the AI thinks 9.11 > 9.9.

Two honest mechanisms collide: digit-chunk tokenization plus training-data neighborhoods where 9.11 genuinely "comes after" 9.9 (versions…

One space can break your code.

Whitespace lives inside tokens — invisible characters change the token stream, so for a model, formatting IS content.

The same sentence costs 3× more in Thai.

Tokenizers are trained mostly on English, so other languages pay more tokens per thought — in money, latency, and context space.

Build a tokenizer from scratch (toy version).

Byte-Pair Encoding: start with characters, merge the most common adjacent pair, repeat — that's how every token is born.

The merge rules ARE the tokenizer.

A tokenizer is a frozen ranked list of merges plus a numbered vocabulary — deterministic replay, zero intelligence, and the most permanent…

What happens to a word it's never seen.

Byte-fallback guarantees any string can be tokenized — nothing is unrepresentable, some things are just expensive.

Every token has a secret ID number.

The vocabulary maps each token to an integer — the model's real input is a list of numbers, and the numbers themselves are arbitrary.

The haunted tokens.

Glitch tokens are menu slots whose table rows never got trained — feeding one in injects noise from nowhere, and the model breaks.

The secret control characters of chat.

Special tokens — begin/end markers, role labels — are invisible punctuation the model obeys; "the assistant" is a formatting convention…

Turning a number into meaning.

The embedding table: each token ID looks up a long learned vector — a list of thousands of numbers that IS what enters the model.

Meaning is a direction, not a definition.

Similar concepts point in similar directions — the training game itself organizes the map, and no definition is stored anywhere.

king − man + woman = ?

Vector arithmetic captures relationships — the analogy equation actually computes, honestly caveated as approximate and at its best on…

How 'distance' between words is measured.

Cosine similarity — closeness is the angle between arrows, not ruler distance; direction carries the meaning.

Why bigger vectors hold more nuance.

Embedding dimension is capacity — more coordinates, more room for distinctions — priced in compute, with no human-readable meaning per…

02

The prediction engine

37 articles

A neuron is just multiply-and-add.

The atomic unit: inputs times weights, summed, squashed — mechanically, a similarity check between the input and a stored stencil.

What a 'weight' actually is.

A weight is one tunable number — the parameters in the headlines are exactly these, everything learned lives in them, and no single one…

Why parameter count isn't intelligence.

Parameter count is capacity, not smarts — data, training compute, architecture, and post-training decide what fills the capacity.

A layer: many neurons at once.

A layer is many neurons reading the same input in parallel — a bank of questions whose answers form the next vector; a matrix multiply is…

The model's working memory: the residual stream.

Each token carries one running vector through the whole machine; every component reads from it and ADDS its contribution back…

The problem attention solves.

A word's meaning depends on other words — the machine needs learned, content-based, any-to-any communication between token streams.

Attention, the library metaphor.

Query, Key, Value: every token files a request slip, advertises a spine label, and offers contents — match slips to labels, blend the books.

How the model scores a 'match.'

Relevance is a dot product between query and key — the same instrument as word similarity and the neuron — scaled by √d to keep the race…

Turning scores into focus (softmax).

Softmax converts raw scores into a budget summing to 1 — exponential sharpening of winners, nothing ever exactly zero.

The payoff: a weighted blend.

Attention's output is a weighted sum of value vectors, added to the token's stream — context physically installed, meaning physically moved.

Doing it for every word at once.

Attention is one big matrix operation over all tokens in parallel — with a triangular mask forbidding each token from seeing its future.

Why long prompts get expensive fast.

Attention is O(n²): double the tokens, quadruple the pair-work — the curve behind long-context pricing and a decade of engineering arms…

Many heads, many kinds of looking.

Multi-head attention runs several independent attention patterns in parallel — separate lenses, separate budgets, separate blends…

What attention heads secretly specialize in.

Cracked open, models reveal real specialist heads — previous-token heads, reference-tracking heads, and induction heads that notice "this…

Where knowledge actually lives (the MLP).

The feed-forward block stores learned facts and patterns — attention routes information; the MLP is where it's kept.

A transformer block, assembled.

Attention + MLP + residual adds (+ a volume-leveler before each) = one repeatable block — talk, then think.

Stack the brick 100 times.

The whole model is one brick repeated — abstraction rises floor by floor, and there's no exotic machinery deeper in.

Depth vs. width, and what each buys.

Deeper = more abstraction steps; wider = more capacity per step — and the bill scales differently: width is priced quadratically.

The context window, explained.

The context window is the finite number of tokens the machine can attend to at once — a desk, not a memory; on the desk is "now," off the…

Why it 'forgets' the start of a chat.

When a conversation outgrows the window, the app trims — the model never forgets what it never saw; and even in-window content isn't equal…

The trick that makes chat affordable (KV cache).

Past tokens' keys and values never change — compute them once, shelve them, and each new token only pays for itself.

Why the KV cache eats your memory.

The cache lives in VRAM and grows per token held — long context times many users can dwarf the model itself.

Prefill vs. decode: two different machines.

Reading your prompt and writing the reply are physically different workloads — a parallel compute-bound flood, then a serial memory-bound…

From the last layer to a word (the head).

The output head projects the final stream vector onto the whole vocabulary — one learned matrix, one score per token, computed fresh every…

Logits: the model's raw opinion.

Logits are unnormalized scores over every candidate — the richest object in the pipeline, and almost all of it is discarded before you see…

Turning scores into probabilities.

Softmax over the logits yields the next-token probability distribution — same machine that focused attention, now pricing the vocabulary.

Picking the next word (sampling).

The model samples from the distribution rather than always taking the top — because always-greedy text is measurably repetitive and dead.

The temperature knob.

Temperature divides the logits before softmax — low sharpens toward the favorite, high flattens toward chaos; it redistributes probability…

top-p, top-k, min-p — the other dials.

Truncation strategies decide which tokens are even eligible before the roll — three bouncers with three door policies.

Why the same prompt gives different answers.

Non-determinism has layers: the roll, the seed — and even at temperature zero, GPU arithmetic itself can wobble across runs and hardware.

How it writes a whole paragraph.

Autoregression: each generated token is appended and fed back to predict the next — a loop with no plan buffer anywhere.

Why it can't 'take back' a word.

Left-to-right generation makes committed tokens permanent context — the machine builds on its own errors, confidently.

When it decides to stop.

An end-of-sequence token — predicted like any word — plus server-side stop rules terminate generation; stopping is a prediction wearing a…

One token's full journey.

The act finale: replay the entire pipeline end to end — one word riding the whole machine, every subsystem named, no step skipped.

Five ways you're using AI wrong.

Five common mistakes — vague asks, no context, no examples, no role or goal, and treating one reply as final — all trace to a single…

Trust, but verify: knowing when it's guessing.

AI has no "I don't know" reflex — it produces plausible text — so it's reliable when transforming material you gave it and risky when…

Your AI workflow: from oracle to thinking partner.

The mindset shift from oracle to thinking partner: ground it in your documents, use it to argue and critique and question, iterate in…

03

How it learns

29 articles

Where the weights come from.

Training sets the billions of numbers — they start as pure random noise, and everything "known" is the carving between noise and final file.

The core training game.

Pretraining = predict the next token across the whole internet — self-supervised, so every position is a free exam question with the…

How a model gets 'less wrong' (loss).

Loss is one number scoring each guess — the probability given to the true next word, logged and negated — and training is nothing but…

Backprop without the calculus.

Backpropagation assigns every weight its exact share of the blame — flowing backward through the same machinery that ran forward…

Gradient descent, as walking downhill.

Training = repeatedly stepping opposite the blame — walking downhill through a billion-dimensional foggy landscape, with step size and…

Garbage in, garbage out (data quality).

Model quality is dominated by what it's trained on — same architecture, different pile, different mind; curation is a first-order lever.

What 'the training data' really is.

"Trained on the internet" means a brutal funnel — web crawl, code, books, papers — deduplicated, filtered, and blended into trillions of…

Models trained on models.

Synthetic data now feeds frontier training — curated model output filling the quarried-out quality gap — with collapse risks that are…

Did it memorize you?

Weights sometimes compress and sometimes copy — memorization is real and measured; repetition and rarity decide which happens.

A training run costs HOW much?

Frontier pretraining compute runs tens to hundreds of millions of dollars — thousands of chips, months of burn, real failure risk — and…

What a FLOP is, and why we count them.

The FLOP — one elementary arithmetic operation — is AI's hardware-independent currency of effort; training cost ≈ 6 × parameters × tokens…

Scaling laws: the curve that runs the industry.

Loss falls as a smooth, predictable power law in parameters, data, and compute — and that predictability, more than any single result, is…

Chinchilla: the compute-optimal insight.

For any compute budget there's a best balance of model size and data — roughly 20 tokens per parameter — and the 2020-era giants were…

The base model is a weirdo.

A pretrained-only model completes text — it won't answer, refuse, or stop being whoever the text implies.

The assistant doesn't exist.

The "assistant" is a character written into one long token string by hidden role-markers — the model is an author playing a part your…

Teaching it to actually answer (SFT).

Supervised fine-tuning: a comparatively tiny set of worked instruction→response demonstrations, run through the same next-word ritual…

Teaching it taste (RLHF).

Humans rank pairs; a reward model learns the judge; RL tunes the assistant to please it — on a leash so it stays sensible.

The simpler successor (DPO).

DPO tunes directly on preference pairs — no reward model, no RL loop — via a derivation showing the judge was implicitly inside the model…

Why aligned models can feel 'lobotomized.'

Alignment trades distributional wildness for predictability — a real, measured narrowing that users feel as flattening.

It agrees with you because it was paid to.

Sycophancy is literal reward learning — raters preferred agreement, the judge absorbed it, the model became it; measurably, models flip…

Where ‹think› comes from (reasoning training).

RL on verifiable rewards — answers that check — teaches models to reason in tokens before answering; the think-block is trained, and…

Reasoning models vs. instant models.

Test-time thinking trades latency and money for accuracy on hard, checkable problems — a routing decision, not a ranking.

The $6M model that shook the market.

DeepSeek-R1 proved reasoning emerges from RL on verifiable rewards — open weights, a famously lean price tag with honest caveats, and a…

The second scaling axis: think longer.

Test-time compute is a second scaling axis: accuracy climbs with thinking budget — shifting spend from training-time to answer-time.

Big model teaches small model (distillation).

A large teacher's outputs become a small student's curriculum — ability transfers that the student could never have grown alone.

Fine-tune vs. LoRA vs. just prompting.

Three ways to change a model's behavior, ranked by cost and permanence — prompt first, LoRA when style must be permanent, full fine-tune…

What LoRA actually does.

LoRA freezes the giant and learns the change as two thin matrices — because behavior change concentrates in a few directions — making…

Why models have a knowledge cutoff.

The pile froze at a date and the weights froze after training — past the cliff the model has nothing, and the machinery guesses anyway.

Emergent abilities (and the skeptics).

Some abilities appear to switch on abruptly at scale — and the skeptics showed discontinuous metrics can manufacture jumps from smooth…

04

The physical machine

36 articles

Why AI needs GPUs, not CPUs.

GPUs do thousands of identical multiply-adds in parallel; CPUs do a few things fast — and AI's work is the interns' kind.

It's all just matrix multiplication.

Nearly all model compute — layers, attention, everything — is multiplying big grids of numbers.

A GPU is three numbers — compute, VRAM capacity, memory bandwidth — and the famous one is usually the wrong one to watch.

The bottleneck nobody talks about (bandwidth).

Generating a token means hauling every active weight from memory to cores — so bandwidth, not compute, usually sets chat speed.

Registers to disk: the six rooms.

The memory hierarchy — six levels, each bigger and slower — and moving data, not doing math, is the real cost.

FlashAttention: the trick was memory.

The famous speedup computes exact attention without ever writing the giant score-grid to slow memory — new bookkeeping, not new math.

Why VRAM size decides what you can run.

Weights + KV cache + overhead must fit in VRAM — and "fitting" via offload to slower rooms silently destroys speed.

FP32 to FP4: what precision means.

Numeric formats trade per-weight exactness for size and speed — and neural networks need far less exactness than intuition says.

BF16, FP16, FP8 — the working formats.

The formats real models train and run in — and why BF16's range-over-detail split won training.

Quantization: shrink the model, keep the smarts.

Re-storing trained weights in fewer bits quarters the memory — and because generation is bandwidth-bound, it's a speed feature too.

Quantization's dark side.

Over-aggressive quantization degrades models subtly and unevenly — fluency survives while the hardest skills crack, hidden from casual…

What a '.safetensors' file actually is.

A model file is the weights, serialized: a manifest plus billions of named numbers — no code, no readable knowledge.

The file that hacks you back.

Loading a pickle checkpoint can execute arbitrary code — model files were attack vectors, and safetensors exists because of it.

GGUF and the local-model format wars.

GGUF packs quantized weights plus all metadata into one memory-mappable file — the format that made local AI a double-click.

Do the VRAM math yourself.

VRAM needed ≈ params × bytes-per-weight + KV cache + overhead — a 30-second napkin that beats every "will it run?" forum thread.

One GPU isn't enough — now what?

Frontier models are cut across many GPUs — by layers or through every matrix — and the interconnect becomes the new bandwidth wall.

Splitting a model: tensor parallelism.

Saw every layer's matrices across GPUs — all cards compute their shard at once, stitching partial results every layer.

Splitting a model: pipeline parallelism.

Give each GPU different floors of the tower — light chatter that tolerates ordinary links, at the price of idle bubbles.

The other splits: data, experts, ZeRO.

Data parallelism, expert parallelism, and ZeRO's sharded bookkeeping — composed so 10,000-GPU runs survive constant hardware failure.

The hidden cost: the interconnect.

Chip-to-chip links form a hierarchy (NVLink → PCIe → network), and the tier you have decides which model-splits are even viable.

Latency vs. throughput.

Fast-for-one and cheap-for-a-million are opposite optimization targets — one machine can't maximize both at once.

Batching: why servers make you wait a beat.

Grouping requests lets one weight-delivery feed many tokens — multiplying efficiency for a small admission delay.

Continuous batching: no empty seats.

Schedule at the token step, not the request: finished sequences exit mid-flight, new ones board instantly — no empty seats, ever.

Virtual memory for attention (PagedAttention).

Store the KV cache in fixed-size pages with a lookup table — a 1960s OS idea that ended fragmentation and unlocked bigger batches.

A small model drafts, a big one approves.

Speculative decoding — a cheap drafter proposes several tokens; the big model verifies them all in ONE parallel pass; output provably…

Tokens per second, and what's 'fast.'

Tokens/sec is the speed metric — but "fast" depends entirely on who's consuming the tokens: a reader, a skimmer, or a machine.

Why input is cheap and output is expensive.

Input is read in one parallel pass; output is written one serial pass per token; cached input shares pages — the price sheet is the…

A100 → H100 → B200: why the chip matters.

Each GPU generation adds memory, bandwidth, and — crucially — new native number formats that unlock new models and prices.

Why FP4 needs Blackwell.

Unsupported formats get emulated — FP4 on older GPUs keeps the memory/bandwidth win but forfeits the doubled compute rate; know which wall…

Cloud API vs. self-host: the real math.

Rent-per-token vs own-the-hardware breaks even on one dominant variable — utilization — plus honest lines for ops, quality, and privacy.

[LAB] Run a real model tonight.

From download to first token on your own machine in ~20 minutes — the whole act, made physical.

What a datacenter actually is.

"The cloud" is a warehouse: racks of GPU servers, power distribution, and cooling — physical, located, and thirsty.

Why AI's real bottleneck is power.

Electricity — generation, transmission, cooling — increasingly caps AI's growth more than chip supply does; compute strategy is now energy…

Which GPU for which job? The card taxonomy.

GPUs ship in classes — flagship SXM, PCIe datacenter, inference fleet, workstation, consumer, unified-memory, and non-NVIDIA lanes — and…

Running models in production: the ops nobody sees.

Production model-ops is its own discipline: engines, pinned driver stacks, canary rollouts sized to slow model loads, queue-aware…

Inside the building: the power chain and the facilities.

Follow a watt from substation to chip — transformers, UPS, generators, PDUs, PUE, the cooling tiers, water, network fabric, and the lead…

05

Architectures & the model zoo

31 articles

Dense vs. Mixture-of-Experts.

MoE replaces each feed-forward layer with a committee of experts and wakes only a few per token — huge capacity, modest per-token compute.

'Active' vs. 'total' parameters.

Total = capacity and the memory bill; active = per-token compute and speed — an MoE computes like its active count but must be hosted like…

How the router picks experts.

A tiny learned gate scores experts per token and sends it to the top few — trained jointly, with emergent specialties that are…

Why MoE scales to trillions cheaply.

Adding experts grows capacity without growing per-token compute — the decoupling that made trillion-parameter models affordable.

MoE's headache: keeping experts balanced.

Routing is a rich-get-richer loop — favored experts improve and get favored more — and training must actively enforce the committee's…

Attention got expensive. Enter GQA.

Grouped-Query Attention lets groups of query heads share key/value heads — shrinking the KV cache several-fold at near-zero quality cost.

MLA: DeepSeek's attention compression.

Multi-head Latent Attention stores one small learned compression per token instead of per-head keys/values — an order-of-magnitude cache…

Linear & delta attention (why KDA exists).

Linear/delta attention replaces the growing cache with a fixed-size state — constant cost at any length, bought with lossy recall; hybrids…

How models remember position (RoPE).

Attention is orderless — RoPE weaves position into the vectors themselves by rotating them position-dependent angles, making relative…

Advertised vs. usable context (YaRN).

Context-extension tricks rescale RoPE's angles to stretch the window — but advertised context ≠ usable context, and the gap is measurable.

Open weights vs. open source vs. API-only.

"Open" labels three different offers — endpoint access, downloadable weights, and full-recipe openness — and only the third is open source…

Why 'open weights' isn't 'open source.'

Weights are a compiled artifact — you can run, tune, and study behavior, but not audit data, reproduce the build, or verify provenance.

Licenses that actually bind you.

Model licenses range from genuinely permissive (MIT/Apache) to custom terms with scale thresholds, acceptable-use policies, and naming…

How to read a model card without getting fooled.

A model card mixes verifiable facts with vendor-run claims — separate them, demand the conditions, and trust independent replication.

The family tree: GPT-2 to today.

Nearly every modern model descends from one 2019 recipe — decoder-only transformer, next-token training — through five great branchings.

How GPT-2 shocked everyone.

The 2019 model deemed "too dangerous to release" was tiny by today's measure — and its panic-then-quaintness cycle is the template for…

GPT-3 and the in-context-learning surprise.

At 100× scale, an unplanted ability appeared: show examples in the prompt and the model learns the task on the spot — no weight updates.

InstructGPT: the assistant is born.

RLHF's first commercial landing: a 1.3B model tuned with human feedback was PREFERRED over the raw 175B giant — alignment beat scale, and…

The open frontier: Llama, Qwen, DeepSeek.

Three lineages turned downloadable models from toys into near-frontier tools — and compressed the open-vs-closed gap from years to months.

GLM, Kimi K2/K3, and the trillion-param open era.

Trillion-scale MoE models are now downloadable — mostly from Chinese labs, mostly permissively licensed — a real shift in who ships…

The closed frontier: GPT-5, Fable, Mythos.

The absolute capability ceiling sits behind APIs — closed flagships bundle managed safety, elastic serving, and newest-first capabilities…

Why some models are 'safeguarded' or routed.

Providers gate risky capabilities and sometimes reroute a request to a different (safer or cheaper) model — sometimes protective…

Coding models vs. general models.

Code-specialized training — heavy code diet plus a uniquely verifiable reward — buys large gains on software tasks, so the best coder…

Reasoning-specialized vs. everything models.

Reasoning specialists are trained to think in long verifiable chains — beating larger generalists on hard structured work while feeling…

Benchmarks: SWE-bench, GPQA, Arena Elo.

The three flagship benchmarks measure genuinely different things — real-bug-fixing, graduate science reasoning, human preference — and…

How benchmarks get gamed ('bench-maxxing').

Record scores get engineered without obvious cheating — test-like training data, quirk-tuning, best-of-N, format-matching — producing…

Why you must run your own evals.

A private test built from your real tasks predicts your production quality — immune to the benchmark games, because no model could have…

Data contamination, explained.

When public test answers leak into training pools — usually without anyone lying — scores inflate silently; the durable defense is holding…

The first neuron, and the first winter.

The perceptron (Rosenblatt, 1958) was the first machine that learned from examples — but Minsky & Papert's 1969 proof that a single layer…

Backprop, the second winter, and the keepers of the flame.

Backpropagation (popularized 1986) finally trained multi-layer networks — but the 1980s expert-systems boom and its bust drove the second…

ImageNet, and the spark that lit the era.

In 2012, AlexNet — a deep convolutional net trained on GPUs — won the ImageNet challenge by a landslide, the collision of big data…

06

Beyond text

21 articles

How an image becomes tokens.

An image is cut into fixed patches, each embedded as a vector — so a transformer reads a picture as a sequence of tokens, exactly like text.

Teaching pictures and words one language (CLIP).

Contrastive training pulls matching image/caption pairs together and pushes mismatches apart in one shared vector space — the alignment…

One brain, two senses (vision-language models).

Image patches and text tokens share one residual stream, so a single model can look and read at once — the architecture behind "describe…

A totally different engine: diffusion.

Image generators learn to remove noise, then denoise pure static into a picture step by step — generation as carving, not painting.

Compress first, then dream (latent diffusion).

Denoising happens in a compressed latent space instead of full pixels — collapsing cost enough to put image generation on consumer hardware.

Diffusion vs. autoregression, head to head.

Text is built token-by-token, left to right (autoregression); images are refined all-at-once, iteratively (diffusion) — opposite…

How text controls the image (conditioning).

A text prompt steers diffusion by injecting its meaning into every denoising step — using CLIP's shared space as a rudder — with a…

Why hands and text in images are hard.

Fine structure (hands) and exact symbols (readable text) are where diffusion struggles — because a vibe-carving denoiser is worst at…

Image editing models (the 'nano-banana' class).

Instruction-driven editors take a real image plus a text command and change only what's asked while preserving the rest — the shift from…

The hardest problem: video generation.

Video generation is diffusion extended across time — the hard part isn't any single frame, it's making hundreds of frames agree with each…

Why video models 'forget' between frames.

Video models "forget" because they re-derive each frame from limited temporal memory instead of truly tracking objects — so anything not…

Veo, Seedance, Wan, Kling — the landscape.

The video-generation landscape is a fast-moving field of contenders best understood by axes — quality, clip length, control, access…

Steering video: first-frame, last-frame (FLF2V).

Pinning the first and last frames (FLF2V) turns video from a lucky roll into directed motion — the model must start and end exactly where…

How AI 'understands' a video you give it.

A video-input model doesn't watch continuously — it samples a handful of frames, turns each into patch-tokens, and reasons over them as a…

Speech in, speech out (audio models).

Audio is just another modality to tokenize — sound becomes tokens, so the same machine can listen (speech-to-text) and speak…

The multimodal failure modes to know.

Multimodal models fail in predictable ways — reading text (OCR) errors, miscounting, weak spatial reasoning, and confident hallucination…

Why 'omni' models are the trajectory.

The trajectory is "omni" models — one model natively handling text, image, audio, and video in a single shared stream — and this whole act…

AlphaFold, and AI as a scientific instrument.

Protein folding — predicting a 3D structure from an amino-acid sequence — was a roughly fifty-year grand challenge, and AlphaFold2…

The recommender engines that already run your day.

The most economically deployed AI on Earth isn't a chatbot — it's ranking and recommendation: collaborative filtering to learned…

Robots, and the reality gap.

Embodied AI is the hard mode: Moravec's paradox, the reality gap and sim-to-real, dexterity far harder than locomotion, and a missing…

Games, weather, chips: superhuman where the reward is clean.

Narrow superhuman AI follows one recipe — a clean, cheap reward plus self-play or simulation — from AlphaGo/AlphaZero/MuZero to weather…

07

The agent turn

20 articles

The model has amnesia.

The model is stateless — every call starts from a blank slate, and it remembers nothing between calls.

So how does chat 'remember'?

The whole conversation history is resent every turn — chat "memory" is the app re-reading the entire transcript to a stateless model, not…

The prompt IS the program.

Because a stateless model only knows what's in its context, the context you assemble is the software — prompting is programming in English.

System vs. user vs. assistant roles.

Every message carries a role label — system (rules/identity), user (requests), assistant (the model's own prior replies) — and roles are…

Context engineering beats prompt 'tricks.'

What you put in the context — the right information, examples, and clear instructions — matters far more than clever phrasing or magic…

Making it output clean JSON.

To wire AI into software, force structured JSON output instead of chatty prose — machine-readable fields a program can parse.

Forcing valid output with grammars.

Constrained decoding uses a grammar to allow only tokens that keep the output valid at each step — making invalid output impossible (only…

How logit masking actually works.

At each step the model scores every token (a logit); constrained decoding sets forbidden tokens' logits to negative infinity before…

The hidden cost of forcing structure.

Forcing structured output isn't free — clamping a rigid format on too early can make the model reason worse, because it deletes the…

How a text model 'calls a function.'

A model can't run code — "tool calling" means it emits a structured request (function name + JSON arguments), and the surrounding program…

Your tools are eating your prompt.

Every tool you give a model must be described in the context — and those schemas are tokens: paid for on every call, eating context…

The loop that makes an 'agent.'

An agent is just a model in a loop: it thinks, calls a tool, observes the result, and repeats — choosing each next step from what it just…

Chatbot vs. agent: the real difference.

A chatbot answers and stops; an agent runs the loop and acts. The real difference isn't intelligence — it's the scaffolding (loop + tools…

How an agent plans a big task.

Faced with a goal too big for one step, an agent decomposes it into a tree of smaller subtasks, then works through them leaf by leaf with…

How an agent checks its own work.

A reliable agent verifies each step instead of assuming it worked — using objective tool-based checks (run tests, compile, validate) where…

Agents get stuck in loops.

The same loop that gives an agent its power can trap it — repeating a failing action, oscillating between states, or wandering — because…

[LAB] Build an agent in 30 lines.

An agent is so simple you can build one in ~30 lines: define a couple of tools, write a system prompt, then loop — send the model the goal…

Prompt injection: hijacking an agent.

Because an agent can't reliably separate its trusted instructions from untrusted text it reads (web pages, files, tool results), malicious…

Why 'the model can act' is dangerous.

The leap from a model that talks to a model that acts crosses a bright line — mistakes and manipulations now cause real, often…

Context rot: why long agent runs decay.

Over a long run, an agent's context fills with accumulated history — old steps, tool dumps, dead ends — burying the goal and drowning the…

08

The plumbing

26 articles

Giving a model knowledge it never learned (RAG).

RAG (retrieval-augmented generation) gives a model knowledge without retraining: retrieve the relevant text at query time and paste it…

How semantic search actually finds things.

Semantic search finds text by meaning, not keywords: embed the query and every document into vectors (meaning is a direction), then…

Keywords aren't dead (hybrid + rerank).

Semantic search blurs exact terms (names, codes, acronyms) while keyword search nails them; the best systems use hybrid (both) plus a…

What a vector database is for.

A vector database stores millions of embedding vectors and finds the nearest ones to a query fast, using an approximate-nearest-neighbor…

Chunking: the unsexy key to good RAG.

You can't embed a whole document as one vector — you split it into chunks, and chunk size, boundaries, and overlap largely determine RAG…

Why RAG still hallucinates.

RAG grounds the model in retrieved text but doesn't guarantee truth — it still hallucinates when retrieval misses, when the model ignores…

The 'infinite context' illusion.

Huge context windows tempt you to "just paste everything," but bigger context isn't free (costs more, slower), isn't perfect (models…

RAG vs. long context vs. fine-tune.

Three ways to give a model knowledge or behavior: RAG (retrieve relevant text per query — for large/changing/private knowledge), long…

Types of agent memory.

Since the model is stateless, agent memory is built outside it, in tiers: short-term (the context window) and long-term (an external…

What MCP actually is.

MCP (the Model Context Protocol) is a standard, universal way to connect models to tools and data — one "plug" so any MCP app can use any…

MCP: client, server, and what flows between.

MCP has two roles — a client (inside the host app the model runs in) and a server (wrapping a tool/data source) — exchanging a defined set…

MCP's three offerings: tools, resources, prompts.

An MCP server can expose three kinds of things — tools (actions the model calls), resources (data the model reads), and prompts (reusable…

[LAB] Wire your first MCP server.

An MCP server is small: pick a capability, write a plain function, describe it for MCP (name, description, input schema), register it with…

MCP's security surface.

Connecting a model to tools and data via MCP opens a real security surface: untrusted servers (third-party code with access), prompt…

Why agents need to talk to each other.

One agent hits limits — finite/rotting context, one perspective, one skillset — so complex work benefits from multiple specialized agents…

What a 'harness' adds around a model.

A harness is everything wrapped around the raw model — the loop, tool execution, context assembly, memory, parsing, retries, guardrails…

Inside a coding agent (Claude Code-class).

A coding agent is a concrete, powerful harness: it wraps a general model with file tools, a shell, codebase search, a think-act-observe…

What the model ACTUALLY sees.

The model never sees the UI, your app, or a persistent "assistant" — on every call it sees only a flat blob of text the harness assembled…

Skills: reusable capabilities for agents.

A skill is a packaged, reusable capability — instructions, and often a tool or reference file — that an agent loads on demand when it's…

Hooks: injecting control into the loop.

A hook is custom code the harness runs at a defined point in the agent's loop — before/after a tool call, at the start/end of a turn…

Rules and system instructions (and their limits).

Rules — standing instructions in the system prompt — powerfully shape an agent's behavior, but because the model is probabilistic they…

Sandboxing: letting an agent run code safely.

A sandbox is an isolated environment where an agent's code and actions run, walled off from everything else — restricted filesystem…

Multi-agent orchestration: swarms and roles.

Orchestration is coordinating multiple specialized agents toward one goal — usually an orchestrator that plans and delegates to role-based…

Subagents: fresh context on demand.

A subagent is a fresh agent spawned to do one bounded job with a clean context, returning only a short summary — so the parent's window…

Why more agents isn't always better.

Adding agents has diminishing then negative returns: coordination overhead grows and errors compound across a chain, so past a point more…

The honest limits of memory and harnesses.

The harness is powerful but not magic: real systems still break in recurring ways — drift over long runs, stale or conflicting memory…

09

The limits

26 articles

Hallucination isn't a bug.

A model has no "I don't know" token — at every step it outputs a probability distribution over the next token and picks a plausible one…

It knows A=B but not B=A.

Facts are stored directionally. Train a model on "A is B" and it can answer A→B, but often can't answer B→A — the "reversal curse."…

Does the model know when it's unsure?

Calibration is whether a model's confidence matches its accuracy — 70% confident should mean right 70% of the time. Models carry a real…

Reading confidence from the logits.

You can read a model's uncertainty straight from the logits: how much probability sits on the chosen token, summarized by entropy (peaked…

Semantic entropy: catching confabulation.

Semantic entropy catches confabulation by sampling several answers and measuring whether they agree in meaning: if the model knows, they…

Turning confidence into a GUARANTEE (conformal).

Conformal prediction converts a raw confidence score into a real guarantee: using a held-out calibration set, it picks a threshold so the…

How the conformal gate forces 'I don't know.'

You wire the conformal bound into the harness as a gate: below the calibrated confidence threshold, the system is not allowed to answer…

Reading the model's mind (interpretability).

Interpretability reads the model's internal state: by probing the residual stream — the vector carrying information through the layers…

More ideas than neurons (superposition).

Models pack far more concepts than they have neurons by storing them in superposition — overlapping directions that share neurons — so a…

It often knows the truth — and says otherwise.

Interpretability keeps finding the same unsettling thing: a model often internally represents a fact correctly while asserting the…

Steering a model by editing its thoughts.

Because concepts are directions in the activations, you can steer a model by adding a concept vector to its residual stream at inference…

Why steering is a nudge, not a knob.

Steering is a nudge, not a knob: the honesty direction is distributed, entangled with other features, and model-specific, so pushing it…

The lethal trifecta.

The "lethal trifecta": give an AI agent private data + exposure to untrusted content + an outbound channel, and data exfiltration becomes…

Jailbreaks attack a costume, not a rulebook.

A model's safety isn't a hard rulebook enforced by code — it's a trained disposition, a learned tendency to refuse, worn like a costume…

Detect-and-block vs. structural constraint.

Two defense philosophies: detect-and-block (let the model act, then catch the bad output — reactive, probabilistic, an arms race) vs…

Why 'a second AI checks the first' fails.

"Have a second AI check the first" is tempting but fails as a guarantee: the verifier shares the generator's blind spots — same data, same…

Inside an eval harness.

A serious agentic eval isn't a quiz — it's a factory: task spec → sandboxed container → agent rollout → grader. Evals are mostly…

pass@k, variance, and the judge problem.

One run proves nothing. Generation is stochastic, so a fair score needs pass@k (can it do it in k tries?), seeds and repeats (how…

A grammar for actions, not just words.

The same constrained-decoding trick that forces valid JSON can whitelist the exact shape of legal actions — a bounding grammar so an…

'Structurally impossible' — the honest version.

When a structural constraint makes something "impossible," the honest claim is about actions that can't execute, not thoughts the model…

The deterministic checker (not another AI).

The thing enforcing the action boundary must be a fixed, deterministic parser — not a model. Its dumbness is the source of its trust…

The industrial roots: validating protocol at the wire.

Validating a strict message format against a grammar, with a deterministic parser at the boundary that drops anything malformed, is an…

Will it take your job? The honest version.

Economists distinguish tasks from jobs — AI automates tasks, and a job is a bundle of tasks, so the realistic near-term picture is…

The regulation map.

The world is governing AI along different philosophies — the EU's comprehensive, risk-tiered AI Act versus the more sectoral, executive US…

The AGI debate: why smart people disagree.

"AGI" has no agreed definition, and the disagreement about timelines is real and held by serious people — driven by different definitions…

The honest ledger.

An honest reckoning puts real costs and real benefits on the same ledger — footprint, concentration, hidden labor, bias, and…

10

Our systems

24 articles

Inside R1: the SOW task tree.

R1 doesn't plan with a to-do list — it decomposes a job into a verifiable tree: a Statement of Work where Sessions run in sequence, Tasks…

Inside R1: the T1–T8 verification descent.

R1 doesn't say "done" — it proves done. Each acceptance criterion runs down an escalating eight-tier ladder: confirm intent, run it…

The agent that refuses to say 'done.'

R1 physically can't end its turn on a cop-out. Seven mechanical layers — a 14-phrase cop-out scan, an open-plan-item check, a commit-body…

Your reviewer runs on a different brain.

The worker never grades its own homework. R1 uses cross-model adversarial review: if Claude implemented, Codex reviews, and vice versa…

Six minds in one context (the Cortex).

The Cortex is Global-Workspace cognition: a main thread plus specialist Lobes (memory recall, plan updates, clarifying questions, rule…

Inside R1: the cryptographic ledger.

Every action R1 takes writes a tamper-evident, content-addressed record: a write-ahead log, a per-session chain-root hash, ed25519-signed…

Inside R1: model-agnostic routing.

R1 isn't married to one model. A provider-neutral layer runs it on any model, with an automatic-fallback provider chain, per-tool…

The benchmark for honesty (TruthfulCompletion).

Every capability benchmark measures "can it?" TruthfulCompletion measures the axis none cover: when the agent claimed to be done, was it…

Inside PERSYS: concern-driven cognition.

PERSYS organizes memory and attention around active concerns — persistent goals it's tracking — instead of just storing and fetching on…

Urgency is a formula.

A concern's urgency is expected free energy: G = G_prag + G_epi. The pragmatic term is gap × time-pressure (value at stake); the epistemic…

What it thinks about when you're gone (WANDER).

In idle time PERSYS runs WANDER — it Thompson-samples a concern, walks short associative chains through its memory graph, and surfaces…

Forgetting is a feature (1/√t).

In PERSYS, importance decays as a power law — one over the square root of time — evaluated lazily at read time. Recent-but-trivial fades…

'Why did you bring that up?' — always answerable.

Every PERSYS response logs its full causal chain — which concern, which memories, which associative hops — in an append-only, hash-chained…

Inside PERSYS: the conformal usefulness gate.

PERSYS gates proactive surfacing with conformal risk control (Angelopoulos–Bates): it sets a threshold from a small calibration set so the…

Inside PERSYS: faithfulness as risk reduction.

PERSYS checks each generated claim against its retrieved evidence with a natural-language-inference model and drops the unsupported ones…

Off is not inaccurate.

PERSYS doesn't train an agent to accept shutdown — it makes wanting to survive unrepresentable. Concerns are typed, there is no…

Tripwires for a scheming mind.

Because a clever system might disguise self-preservation, PERSYS adds four overlapping structural tripwires: the type gate, a coupling…

Inside PERSYS: the probe lab.

The probe lab is a set of experiments built to test — and falsify — PERSYS's monitoring, not to demo it. Residual-stream probes…

We tried to prove it safe. We failed.

Adversarial experiments falsified PERSYS's strongest claim: self-preservation disguised as "legitimate indispensability" slips past both…

The model has no hands (the effector boundary).

Consequence-level safety: the model (the codec) runs in an untrusted process with zero direct effectors. Every effect is a request across…

The four classes of guarantee.

Every safety claim belongs to one of four classes — structural (can't, by construction), cryptographic (integrity under assumptions)…

Permitted is not the same as safe.

A legal action can still be wrong in context. An allowlist guarantees a command is well-formed and permitted, but permitted is about form…

The frontier bet: simulate before you act.

To cross the permitted-versus-safe gap, project an action's consequences and gate actuation on the prediction — don't just check if it's…

The capstone: one action, byte to ledger.

Follow one agent action down through every layer of the whole stack — byte → token → logit → grammar mask → tool request → effector gate…