SLMs — small as a strategy

Small Language Models — roughly 0.5B to 14B parameters, runnable on consumer hardware — used to be understood as "big models, but worse." That framing died, and you already know each nail in its coffin:

1. Overtraining (Topic 9): Chinchilla-optimal was compute-optimal, not inference-optimal — so labs now train small models on trillions of tokens, thousands of tokens per parameter, making modern 8Bs stronger than 2020's 175B.
2. Distillation (Topic 17): frontier teachers manufacture textbook-quality training data for small students — the Phi lineage's whole thesis.
3. The specialist theorem (Topics 11–12, this course's recurring business case): a fine-tuned 8B beats a general giant on the narrow task it was tuned for, at ~1/50th the cost.

So the modern question isn't "how much worse is the small model?" but "which jobs are small-model jobs?" — and Modules 4–7 already gave you the list: the easy 80% behind a router or cascade (Topic 38's economics); agent steps, where low latency and high per-step reliability beat raw brilliance (Topic 58's compounding math loves fast, dependable steps); draft models for speculative decoding (Topic 35 — an SLM's literal job description); guardrails and classifiers (input filtering, routing decisions, injection screening — sub-second, always-on jobs); and on-device deployment for privacy, offline operation, and zero-network latency (Module 9's subject).

The honest limits, with their standard remedies: SLMs hold less world knowledge (→ pair with RAG — the Module 6 marriage: facts from retrieval, fluency from the model; a 3B with good retrieval beats a 70B recalling from stale weights on your documents), reason less deeply over long horizons (→ decompose via Topic 59's workflows, or route hard cases up), and buckle under 40-rule instruction stacks (→ fine-tune the behavior in instead — Topic 12's rung 4, and precisely where your Module 3 skills earn money).

Names to know: Qwen's 0.5B–7B line, Llama 3.2 1B/3B, Phi, Gemma, SmolLM — a tier moving faster than any other, because every technique in this course (distillation, overtraining, quantization, fine-tuning) compounds hardest at small scale.

---