hawkes.aiKey ideas
- The one idea — Forcing structured output isn't free — clamping a rigid format on too early can make the model reason worse, because it deletes the free-form thinking space and masks away the model's best answer path (the "projection tax").
- How it is shown — The same model answering freely (thinks out loud, gets it right) versus forced into strict JSON from token one (no room to think, lands a worse answer).
- The trap to avoid — Assuming structure is always free, so you clamp the tightest schema from the first token — exactly when the tax bites hardest.
- What it sets up — Structured output isn't only for data — it's how a model asks to DO something.
Everyone assumes forcing structured output is free. It isn't — mask away the model's favorite next words and you can mask away its best answer. Structure has a tax.
The one idea
Forcing structured output isn't free — clamping a rigid format on too early can make the model reason worse, because it deletes the free-form thinking space and masks away the model's best answer path (the "projection tax").
Last group, you forced the model to output perfect JSON — validity guaranteed. But that power has a hidden price: constraining what a model can say can quietly make it reason worse. Forcing structure isn't free. There's a tax worth understanding. Models reason well because they think out loud. Chain-of-thought, working through steps in ordinary text, is where good answers come from, and it needs room to explore. Strict structured output gives none. Demand JSON from the first token, and you've deleted the scratchpad. It goes deeper. Recall logit masking crushes forbidden tokens to zero.
How it works — the demo
The same model answering freely (thinks out loud, gets it right) versus forced into strict JSON from token one (no room to think, lands a worse answer).
But the tokens the model most wanted, its best path to a good answer, may be the ones the schema forbids. Masking shoves it onto a permitted but worse route. That detour is the tax. So two costs. One: no space to think before answering. Two: the model is pushed off its natural phrasing onto whatever the format allows, even when that's weaker. Both mean the same thing: rigid structure, imposed too early, measurably drops reasoning quality. The fix is simple: separate thinking from formatting. Let the model reason freely first, in an unconstrained scratchpad, then structure only the final answer. Or put a reasoning field before the answer fields, so it thinks in text, then fills the slots.
The trap to avoid
Assuming structure is always free, so you clamp the tightest schema from the first token — exactly when the tax bites hardest.
Why it matters — and what’s next
Structured output isn't only for data — it's how a model asks to DO something.
The trap: assuming structure is free, so you clamp the tightest schema from token one. That's when the tax bites hardest; you've caged the reasoning, not the result. Constrain the output, not the thinking. Reserve rigid format for what a program must parse. So forcing structure is powerful but taxed; use it deliberately. And structured output isn't only for data. That same clean JSON is how a model asks to do something: to call a tool. Next: how a text model calls a function.
This is one short episode in AI: Zero → Frontier, a step-by-step climb through how AI actually works. Each episode builds only on the ones before it.
Full transcript 2:30 of narration
Last group, you forced the model to output perfect JSON — validity guaranteed. But that power has a hidden price: constraining what a model can say can quietly make it reason worse. Forcing structure isn't free. There's a tax worth understanding.
Models reason well because they think out loud. Chain-of-thought, working through steps in ordinary text, is where good answers come from, and it needs room to explore. Strict structured output gives none. Demand JSON from the first token, and you've deleted the scratchpad.
It goes deeper. Recall logit masking crushes forbidden tokens to zero. But the tokens the model most wanted, its best path to a good answer, may be the ones the schema forbids. Masking shoves it onto a permitted but worse route. That detour is the tax.
So two costs. One: no space to think before answering. Two: the model is pushed off its natural phrasing onto whatever the format allows, even when that's weaker. Both mean the same thing: rigid structure, imposed too early, measurably drops reasoning quality.
The fix is simple: separate thinking from formatting. Let the model reason freely first, in an unconstrained scratchpad, then structure only the final answer. Or put a reasoning field before the answer fields, so it thinks in text, then fills the slots.
The trap: assuming structure is free, so you clamp the tightest schema from token one. That's when the tax bites hardest; you've caged the reasoning, not the result. Constrain the output, not the thinking. Reserve rigid format for what a program must parse.
So forcing structure is powerful but taxed; use it deliberately. And structured output isn't only for data. That same clean JSON is how a model asks to do something: to call a tool. Next: how a text model calls a function.