Reasoning Models — the course comes full circle

Topic 67 of 90Module 8: Model Types4 min read

The finale, and three flags planted across ten lessons converge here. Recall them: Topic 54 — chain-of-thought works because tokens are compute (each token = one forward pass; reasoning in tokens = buying more passes). Topic 27 — RLVR: replace hackable learned rewards with verifiable ones. Topic 45 — GRPO: sample a group of answers, reward the verifiably correct ones relative to the group, no value model needed.

A reasoning model is what happens when you stop prompting for chain-of-thought and train it in with RL: o1, DeepSeek-R1, Claude's extended thinking, QwQ, Gemini's thinking modes. The recipe, at its cleanest (R1 made it public): take a capable base model; give it math and code problems with checkable answers; let it generate long solution attempts; reward correct final answers (GRPO); repeat at scale. And here is the genuinely remarkable finding — the reasoning behaviors emerge rather than being taught. Nobody wrote demonstrations of self-checking; under nothing but "correct answers score," the model discovers re-examining its work, backtracking ("wait — let me reconsider"), trying alternative approaches, and spending more tokens on harder problems. R1's training logs famously show response lengths growing on their own as the model learns that thinking longer pays. Selection pressure, not instruction.

This creates the field's new scaling axis: test-time compute. For years, capability scaled with parameters and training tokens (Topic 9). Now it also scales with thinking tokens at inference — same weights, more deliberation, better answers, with smooth measurable curves. Modern APIs expose this as a dial (thinking budgets / effort levels), and the industry has converged on hybrid models: one model, toggleable thinking — instant mode for easy queries, deliberation for hard ones.

Two more pieces complete the picture. Reasoning distills (Topic 17, one more encore): R1's release proved that SFT on a big reasoner's thinking traces transfers much of the ability into 7B–32B students — small models visibly deliberating, trained by imitation of deliberation. And the honest cost accounting, in Topic 38's terms: thinking tokens are output tokens — the expensive kind — so reasoning models multiply latency and cost, sometimes 10–50× on hard problems; they overthink trivial queries hilariously (three paragraphs deliberating "what's 2+2"); they help most exactly where their training lived (math, code, logic, planning) and much less on factual recall (weights still don't contain your docs — RAG remains undefeated) or open-ended creative work. The product answer is one you've known since Topic 38: route — deliberation is a premium tier, not a default. One epistemic caveat to carry: the visible thinking text is not guaranteed to faithfully reflect the model's actual computation — research shows the transcript and the true process can diverge. Useful, inspectable, but not a窗 into the soul — the same humility Topic 7 demanded of attention maps.

Summary

Reasoning models = chain-of-thought trained in via RL on verifiable rewards (GRPO), yielding emergent self-checking and backtracking, a new test-time-compute scaling axis, distillable traces — at real token cost, best routed to genuinely hard problems.

Mental model

System 1 and System 2, budgeted: the same mind answering reflexively or sitting down with scratch paper — where the habit of scratch-paper work was instilled purely by grading final answers, and every minute of deliberation appears on the bill.

Mistakes to avoid

  • Defaulting all traffic to a reasoning model "for quality." You'll pay deliberation prices for reflex questions — the router (Topics 38, 59) is where reasoning models belong in an architecture.
  • Reading the thinking transcript as ground truth about the model's process. It's a useful artifact with a known faithfulness gap — evidence, not confession.

Exercise

Run a distilled local reasoner (ollama run deepseek-r1:7b or qwq) against a same-size non-reasoning model on three problems: a multi-step math word problem, a tricky logic puzzle, and a plain factual question. For each, note correctness, thinking-token count, and wall-clock time. You'll observe the entire topic in one sitting: the deliberation win on problems 1–2, the overthinking tax on problem 3 — and you'll have watched emergent backtracking ("wait, actually…") scroll past in your own terminal.

Module 8: complete. The zoo, mapped — and notice what the tour proved: every exotic species decomposed into course fundamentals. Vision = translation into embedding space. Small = overtraining + distillation + specialization. MoE = the FFN warehouse, subdivided. Coding = verification asymmetry, industrialized. Reasoning = CoT + RLVR + GRPO. You didn't learn six new things today; you watched everything you already knew recombine.

Next: Module 9 — Deployment. Shorter and intensely practical: local inference and on-device AI (phones, laptops, and what NPUs actually do), API serving (consolidating your Module 4–5 serving knowledge into deployment shapes), cloud GPUs (the rental market decoded — spot vs on-demand, where to actually rent, cost math), and edge AI (the constraints at the far end). Then Modules 10–11 bring evaluation and the real-world build playbooks that cap the course.