Coding Models — where the world grades homework

Why did code become the field's favorite domain — the place where models improved fastest and products (Copilot, Claude Code, Cursor) matured first? Four reasons, three of which you can already recite:

1. Verification asymmetry, maximal (Topics 58, 27): code has free, instant, objective verifiers — compilers, type-checkers, test suites, execution. That makes it RLVR heaven (reward = tests pass, unhackable) and agent heaven (every step checkable, converting compounding failure into retry-until-verified).
2. Training data abundance: decades of public code with commits, issues, and reviews — process data, not just artifacts.
3. Economic pull: developers pay, measure productivity, and tolerate iteration.
4. The bonus: code in the pretraining mixture improves general reasoning (Topic 18's note) — structured, causal, consequence-laden text turns out to be excellent brain food.

What makes a model a "coding model" — a recipe you fully possess: heavy code in the data mixture, often via continued pretraining at Topic 20 scale (Code Llama = Llama 2 + ~500B code tokens — the sandwich, deployed); long context for repositories; and one genuinely new training format: FIM — fill-in-the-middle. Standard training predicts left-to-right, but real coding means inserting code between existing lines. FIM restructures training examples with special tokens — here's the prefix, here's the suffix, now produce the middle — teaching bidirectional-ish completion within the causal architecture. FIM is what powers every Tab-completion you've ever accepted.

Which surfaces the key product split — two different animals sharing the word "coding model":

	Completion models	Agentic/chat coding models
Job	Tab-complete as you type	"Fix this issue across the repo"
Training	FIM-heavy	Instruction + tool-use + RL on real tasks
Latency budget	<500 ms — SLM territory (Topic 63)	Minutes acceptable — frontier territory
Shape	One fast infill	Topic 58's loop: read, edit, run tests, iterate

The agentic side is trained increasingly on execution feedback — RL over real repository tasks where reward is the test suite going green — which is why the benchmark that matters shifted from HumanEval (small function puzzles, now saturated and contamination-riddled) to SWE-bench (resolve actual GitHub issues in real repos, end-to-end) and contamination-resistant designs like LiveCodeBench that only use problems published after model cutoffs — Topic 18's decontamination principle, weaponized into benchmark design. Names on the open side: Qwen-Coder, DeepSeek-Coder, Codestral.

---