Module 5 · Local AI Ecosystem

Lesson 9The Toolchain — Nine Tools, Properly Placed

9 topics15 min readTopics 39–47

Welcome to Module 5: the Local AI Ecosystem — the lesson where every tool this course has name-dropped gets its proper introduction. Fair warning about what changes here: the theory is done. You already understand GGUF, PagedAttention, LoRA, chat templates, and bandwidth ceilings — so each tool profile today is short on new concepts and long on positioning: what it is, what it's built on, and exactly when you reach for it. Nine tools, one map. Here's the map first:

Nine tools placed on one map: the running layer (llama.cpp, Ollama, vLLM, MLX), the hub (Hugging Face), and the training layer (PEFT, TRL, Axolotl, Unsloth) — with the wraps-this arrows that explain each one's position.
Nine tools placed on one map: the running layer (llama.cpp, Ollama, vLLM, MLX), the hub (Hugging Face), and the training layer (PEFT, TRL, Axolotl, Unsloth) — with the wraps-this arrows that explain each one's position.

Part 1 — Running Models

In this lesson

  1. 39llama.cppthe engine everything else wrapsThe origin story is genuinely important for understanding the ecosystem: in March 2023, Georgi Gerganov spent a weekend porting Meta's…2 min
  2. 40Ollamathe Docker of local modelsIf llama.cpp is the engine, Ollama is the product: one install, then models become as easy as containers. The Docker analogy isn't loose —…2 min
  3. 41vLLMTopic 37, installableYou already know vLLM's soul: PagedAttention and continuous batching — it's the reference implementation of half of Lesson 8. The practical…2 min
  4. 42MLXyour M4's native tongueMLX is Apple's own ML framework, built by Apple researchers specifically for Apple Silicon — and it exists because of the architecture you…2 min
  5. 43Hugging Facethe town squareNot a tool — the place. Every topic in this course has quietly routed through it: models pulled, datasets loaded, tokenizers inspected. Its…2 min
  6. 44PEFTthe adapter libraryYou know the methods (Topics 22–23); PEFT is Hugging Face's implementation of them, and you've already called it (LoraConfig,…2 min
  7. 45TRLthe trainer zooTRL ("Transformer Reinforcement Learning") is HF's official home of stages 2 and 3 — the library whose SFTTrainer ran your capstone. Its…2 min
  8. 46Axolotlfine-tuning as configurationEverything so far scripts training in Python. Axolotl takes the infrastructure-as-code stance: an entire fine-tune — base model, dataset…2 min
  9. 47Unsloththe speed layerThe promised demystification. Unsloth makes QLoRA fine-tuning roughly 2× faster with dramatically less VRAM than the vanilla stack — and…3 min

Start with Topic 39: llama.cpp, or browse all 90 topics.