LLM Engineering Reference
Every topic in Modern LLM Engineering From Scratch, as its own page. Read one when you need one, or take them in order as a course.
Module 1Foundations
What an LLM is, and what it can see.
- 1LLM Basicsthe autocomplete that ate the internetForget "artificial intelligence" for a second. An LLM (Large Language Model) is, at its core, a very powerful…2 min
- 2How AI Models Workknobs, guesses, and correctionsHow does a model learn to predict the next word? Three ideas: Idea 1: Everything is numbers. Computers can't…3 min
- 3Tokensthe model's alphabetHere's something surprising: models don't read words. They don't read letters either. They read tokens —…2 min
- 4Tokenizationhow the chunks get chosenSo who decides that "unbelievable" splits into un/believ/able? A separate small program called the tokenizer,…2 min
- 5Context Windowsthe model's working memoryThe context window is the maximum number of tokens a model can look at at once — your entire conversation,…3 min
- 6Embeddingsmeaning as coordinatesIn Lesson 1, text became token IDs: [9906, 445, ...]. But an ID is just an index — token 9906 isn't "more"…3 min
- 7Attentionthe mechanism that made everything possibleI'm teaching attention before transformers, because a transformer is just attention plus plumbing. Understand…4 min
- 8Transformersthe full architectureNow assemble the machine. The transformer (from the 2017 paper "Attention Is All You Need") is what happens…3 min
- 9Parametersthe knobs, quantifiedTime to make "billions of knobs" concrete, because parameter count drives everything you'll do practically:…3 min
- 10Training vs Inferencebuilding the brain vs using itTwo completely different modes of a model's existence. Confusing them causes more beginner misconceptions…3 min
- 11Open-Source vs Closed-Source Modelsthe ecosystem mapLast foundation: the landscape you'll build in. Closed-source (API models): GPT, Claude, Gemini. You send a…4 min
Module 2Datasets & Training
Data is the specification.
- 12Fine-Tuning Basicswhen to reach for it (and when not to)Fine-tuning = taking an already-trained model and continuing training on a small, focused dataset, so the…3 min
- 13SFT Datasetsthe raw materialAn SFT dataset (Supervised Fine-Tuning dataset) is a collection of demonstrations: here's an input, here's…3 min
- 14Instruction Tuningwhy the transformation worksInstruction tuning is the specific, most important flavor of SFT: training on (instruction → response) pairs…3 min
- 15Dataset Formattingwhere fine-tunes silently dieThe least glamorous topic in this module, and the cause of the majority of "my fine-tuned model is broken"…3 min
- 16Preference Datasetsteaching "better," not just "good"SFT has a structural blind spot: it only ever shows positive examples. But much of what makes a model good is…3 min
- 17Synthetic Datasetsmanufacturing your training dataEverything so far assumed you have examples. Usually you have fifty, and need five thousand. The modern…5 min
- 18Data Curationthe editor-in-chief jobCuration and cleaning get used interchangeably, but they're different jobs. Curation = deciding the…4 min
- 19Dataset Cleaningscrubbing the dirtNow the proofreader's job. Real datasets — scraped, synthetic, or human-written — are dirty in predictable…3 min
- 20Continued Pretraininghow knowledge actually enters weightsTwice now I've told you "fine-tuning can't reliably add facts" (Topics 12 and 14). Fair question: then how…4 min
- 21Hallucination Reductionengineering for truthThe module's capstone, where planted threads pay off. First, the mechanical truth about what hallucination…6 min
Module 3Fine-Tuning Methods
Training on hardware you own.
- 22Adapter Tuning & PEFTthe "don't touch the walls" ideaThe family name for everything in this lesson is PEFT: Parameter-Efficient Fine-Tuning. One shared insight…3 min
- 23LoRAthe low-rank trick, properly understoodLoRA (Low-Rank Adaptation) rests on one empirical discovery about fine-tuning itself: When you fully…4 min
- 24Quantizationshrinking the bytesLoRA attacked the trainable parameter side of the memory bill. Quantization attacks the other side: bytes per…4 min
- 25QLoRAthe democratization eventNow snap the two pieces together. LoRA left one stubborn memory item: the frozen base still sits in GPU…3 min
- 26Model Checkpointssave points and how to choose oneLast topic — humbler than the others, but it's the difference between training runs that produce a good model…4 min
- 27RLHFthe heavyweight that started it allRLHF (Reinforcement Learning from Human Feedback) is the stage-3 method that turned GPT-3 into ChatGPT —…4 min
- 28DPOthe shortcut that took overIn 2023, a paper arrived with a title that summarizes itself: "Direct Preference Optimization: Your Language…4 min
- 29GGUFthe format your model wears to leave homeEverything so far lives in the training world: Hugging Face repos, a folder of files — model.safetensors…6 min
Module 4Inference & Optimization
Why it's slow, and the fixes.
- 30GPU Basicsan army of line cooksWhy GPUs at all? Because a transformer's work is overwhelmingly matrix multiplication — millions of tiny…3 min
- 31VRAM Basicsthe budget everything lives insideVRAM is the GPU's own memory, physically soldered next to the chip, connected by a firehose (that 1–3 TB/s…3 min
- 32KV Cachethe flag comes downTime to pay off Lesson 2's flag: generation runs the whole model once per token. Here's the horror show that…4 min
- 33Flash Attentionthe n² flag comes downNow the older flag, planted in Topic 7: attention compares every token with every other token — O(n²). For…4 min
- 34Inference Optimizationthe one insight that organizes everythingSynthesis time. Everything in this lesson — starved cooks, the pantry, the growing minutes, the cabinet walks…5 min
- 35Speculative Decodingcheating the one-token lawThe one-token-per-step law looks unbeatable: token N+1 depends on token N, so generation is inherently…4 min
- 36Batch Inferencethe free throughputNow the biggest prize hiding in Topic 34's physics. Per decode step, the GPU reads all 4.5 GB of weights to…4 min
- 37Model Servingthe restaurant around the chefA model is a chef; a serving engine is the entire restaurant — host stand, order queue, table management,…4 min
- 38Latency vs Quality Tradeoffsthe decision layerFinal topic of the module, and the one that turns physics into product judgment. Every deployment lives…5 min
Module 5Local AI Ecosystem
Nine tools, one map.
- 39llama.cppthe engine everything else wrapsThe origin story is genuinely important for understanding the ecosystem: in March 2023, Georgi Gerganov spent…2 min
- 40Ollamathe Docker of local modelsIf llama.cpp is the engine, Ollama is the product: one install, then models become as easy as containers. The…2 min
- 41vLLMTopic 37, installableYou already know vLLM's soul: PagedAttention and continuous batching — it's the reference implementation of…2 min
- 42MLXyour M4's native tongueMLX is Apple's own ML framework, built by Apple researchers specifically for Apple Silicon — and it exists…2 min
- 43Hugging Facethe town squareNot a tool — the place. Every topic in this course has quietly routed through it: models pulled, datasets…2 min
- 44PEFTthe adapter libraryYou know the methods (Topics 22–23); PEFT is Hugging Face's implementation of them, and you've already called…2 min
- 45TRLthe trainer zooTRL ("Transformer Reinforcement Learning") is HF's official home of stages 2 and 3 — the library whose…2 min
- 46Axolotlfine-tuning as configurationEverything so far scripts training in Python. Axolotl takes the infrastructure-as-code stance: an entire…2 min
- 47Unsloththe speed layerThe promised demystification. Unsloth makes QLoRA fine-tuning roughly 2× faster with dramatically less VRAM…3 min
Module 6RAG & Memory
Giving the model a library and a past.
- 48RAGthe open-book examRAG (Retrieval-Augmented Generation) answers the question every product hits in week one: how does the model…3 min
- 49Chunkingthe cut decides the qualityChunking looks like a preprocessing footnote. It is, in practice, the highest-leverage decision in most RAG…3 min
- 50Vector Databasessearch at scaleYour Topic 48 exercise compared a query against 30 vectors by brute force — checking every single one. Fine…4 min
- 51Semantic Searchwhat embeddings miss, and the hybrid fixSemantic search — embed query, find nearest chunks — has been our quiet workhorse since Lesson 2. Time to be…3 min
- 52Retrieval Pipelinesfrom demo to productionNaive RAG — embed, top-5, stuff, pray — demos beautifully and plateaus at "mediocre with confidence."…4 min
- 53AI Memory Systemsthe goldfish gets a pastClose your eyes and recall Topic 5: the model is stateless — a goldfish handed a transcript each time — and…5 min
Module 7Agents & Workflows
From words to actions.
- 54Prompt Engineeringthe craft, systematizedYou've been prompting for ten lessons; now let's make it an engineering discipline. Start with why prompting…4 min
- 55System Promptsthe product's constitutionEvery technique above, made persistent. The system prompt is the privileged message that precedes every…3 min
- 56Function Callingthe mechanismHere is the bridge between everything so far (text in, text out) and everything agents do — and it rests on…3 min
- 57Tool Callingthe practiceThe mechanism is a solved problem; the craft is where systems succeed or fail. Four disciplines: 1. Tool…4 min
- 58AI Agentsthe loop, demystifiedEverything converges here, and the demystification is total: An agent is a while-loop around tool calling.…5 min
- 59Agentic Workflowsthe pattern library between one prompt and full autonomyBefore reaching for a full agent, reach for a workflow: control flow written in code, with the model doing…4 min
- 60Multi-Agent Systemsteams, and the coordination taxGive each worker in pattern 4 its own full loop, tools, and system prompt, and you've crossed into…4 min
- 61Browser Agentsthe final examEverything in this module — tools, loops, compounding, injection, context budgets — gets stress-tested…5 min
Module 8Model Types
The zoo, decomposed into fundamentals.
- 62VLMshow images become tokensA transformer processes a sequence of vectors. It does not care what those vectors originally were. That…3 min
- 63SLMssmall as a strategySmall Language Models — roughly 0.5B to 14B parameters, runnable on consumer hardware — used to be understood…3 min
- 64Dense Modelsthe honest baselineA short but load-bearing topic. A dense model is one where every parameter participates in every token's…2 min
- 65MoEthe warehouse with a triage deskMixture of Experts answers Topic 64's closing question with surgical precision. The surgery site is exactly…4 min
- 66Coding Modelswhere the world grades homeworkWhy did code become the field's favorite domain — the place where models improved fastest and products…3 min
- 67Reasoning Modelsthe course comes full circleThe finale, and three flags planted across ten lessons converge here. Recall them: Topic 54 —…4 min
Module 9Deployment
The five places models live.
- 68Local Inferencefrom running models to *shipping* themYou run models locally every week now — Ollama, MLX, GGUF, the Topic 34 formula. So this topic addresses the…3 min
- 69On-Device AIphones, NPUs, and the OS as router"On-device" sounds like "local, smaller" — but phones are a genuinely different regime, ruled by three…3 min
- 70API Servingthe production shellYou know the engine (vLLM, Topic 37/41) and the economics (Topic 36). What Module 4 didn't cover is the shell…3 min
- 71Cloud GPUsthe rental market, decodedEverything self-hosted or trained runs on rented silicon, and the rental market has a structure worth…3 min
- 72Edge AI Basicsthe far end of the spectrumPast phones lies edge AI: inference on hardware at the data source — cameras, sensors, robots, kiosks,…4 min
Module 10Evaluation
Measure it, don't vibe it.
- 73AI Benchmarksreading the public scoreboardA benchmark is a standardized exam: a fixed dataset + a metric + a protocol, so any model can be scored…3 min
- 74Human Evalsthe gold standard, with tarnishFor everything with ground truth, machines grade (next topic). But helpfulness, tone, writing quality, "which…3 min
- 75Quality Benchmarkingyour evals, the real skillThe module's center of gravity. Public benchmarks rank the field; human evals are slow and golden; what ships…4 min
- 76Speed Benchmarkingthe stopwatch, held correctlyYou've measured speed since Topic 34's exercise. This topic upgrades the stopwatch into methodology, because…3 min
- 77Cost-per-Token Analysisthe invoice, decodedThe final measurement: money. Token prices look simple — $/million in, $/million out — but a product's…5 min
Module 11Real-World Building
The build playbooks.
- 78Building Chatbotsthe canonical assemblyThe most-built, most-commoditized, most instructive product shape — because a good chatbot is the entire…3 min
- 79Building AI Copilotsintelligence inside the workflowA chatbot is a destination; a copilot is a passenger — it lives inside an existing tool (editor, inbox, CRM,…3 min
- 80AI Automationhumans mostly out of the loopChatbots and copilots serve a present human. Automation runs while nobody watches: ticket triage, document…3 min
- 81AI SaaS Workflowsthe business shapesThe engineering is settled; now the commercial container. Four shapes: AI-native products (the AI is the…3 min
- 82AI Coding Workflowsusing the tools like a professionalModule 8 explained why coding models got good; this topic is the user side — the workflow discipline that…3 min
- 83AI Orchestration SystemsModule 7 at production scaleOne workflow is a script; a company's AI runs hundreds of workflows, thousands of executions a day, for…3 min
- 84AI Product Thinkingthe judgment layerThe final topic of the course — no mechanisms, only judgment: what to build, and the meta-skills that survive…4 min
Module 12The Career Layer
Bonus — converting work into leverage.
- 85Proof-of-Workwhat "personal brand" actually means for engineersDelete the influencer image. For engineers, personal brand = discoverable evidence of ability. When a YC…3 min
- 86The Advanced Project Portfoliowhat to actually buildThe heart of your request. The rule before the list: one flagship done to completion beats ten half-projects…4 min
- 87Open-Source Strategybecoming a known nameYou already understand OSS as a contributor; this topic is OSS as career architecture. Two ladders, climb…3 min
- 88Building Products as an Engineerthe second résuméModule 11 taught product shapes; this topic is why an engineer chasing SV jobs should ship products anyway —…3 min
- 89Distributionwriting and launching like an engineerArtifacts that no one sees don't exist. Distribution is the multiplier on everything above — and engineers…3 min
- 90Landing the YC / a16z / Silicon Valley Jobthe mechanicsNow the target itself. First, understand how these companies actually hire, because it's structurally…6 min