Unsloth — the speed layer

The promised demystification. Unsloth makes QLoRA fine-tuning roughly 2× faster with dramatically less VRAM than the vanilla stack — and the "magic" is Lesson 7's lesson applied to training: hand-written GPU kernels (in Triton) that fuse operations and cut memory traffic in the transformer's hot paths, plus manually-derived gradient math. Fewer trips to the cabinet, exact same arithmetic — like Flash Attention, it's exact, not approximate: the loss curves match vanilla to the decimal.

The API wraps what you know:

from unsloth import FastLanguageModel
 
model, tok = FastLanguageModel.from_pretrained(
    "unsloth/Qwen2.5-7B-Instruct-bnb-4bit", load_in_4bit=True)
model = FastLanguageModel.get_peft_model(model, r=16, lora_alpha=32)
# ...then your exact Lesson 6 SFTTrainer code runs on top, unchanged

It's compatible with TRL, not a replacement — the same trainers, accelerated. Its other flagship feature closes the Lesson 6 loop with a satisfying click:

model.save_pretrained_gguf("mymodel", tok, quantization_method="q4_k_m")

The whole merge → convert → quantize ceremony from Topic 29, one call. Add first-class Colab notebooks for every major model, single-GPU focus (its free tier's deliberate niche — exactly where individuals live), and GRPO support (reasoning experiments on one GPU), and you get its ecosystem position: the default on-ramp for solo fine-tuners. The reason this course taught raw TRL first is now inspectable: Unsloth automates steps you can name — which means when something breaks, you can debug it.

Summary

Fused-kernel acceleration of the standard QLoRA/TRL pipeline — ~2× faster, far less VRAM, bit-exact — with one-call GGUF export; the individual fine-tuner's default.

Mental model

The same Lesson 6 assembly line after a Formula 1 pit crew rebuilt every station — identical product, half the time.

Mistakes to avoid

Assuming speed came from approximation and distrusting the output (it's exact — take the free lunch, same as Flash Attention), and expecting free multi-GPU (that's Axolotl/raw-stack territory).

Exercise

Re-run your Lesson 6 capstone through an Unsloth Colab notebook for your base model — same dataset, same hyperparameters. Record: training time, peak VRAM, final eval loss versus your original run. Two of those numbers should improve; one should match. Knowing which is the whole point of this module.

The Decision Table

The module, compressed into the card you'll actually use:

Situation	Reach for
Run a model on my machine, now	Ollama
Max performance / training on a Mac	MLX
Serve real concurrent users on GPUs	vLLM
Edge devices, custom quants, benchmarking	llama.cpp direct
Bulk offline generation/classification	vLLM offline mode
First fine-tune, one GPU/Colab	Unsloth
Reproducible/team/multi-GPU fine-tunes	Axolotl
Custom training logic, full control	raw PEFT + TRL
Everything above finds its models/datasets	Hugging Face

And your personal stack, given your hardware: MLX + Ollama locally on the M4, Unsloth on Colab (or a rented 24 GB GPU) for training, vLLM the day something you built needs to serve strangers.

Module 5: complete — and with it, the entire "models" half of this curriculum. You can now understand, train, compress, and run LLMs with every major tool placed on one map.

Next: Module 6 — RAG & Memory. The course pivots from shaping models to building systems around them: RAG in full (the "facts" half of Topic 12's mantra, finally getting its own module), vector databases (Lesson 2's embeddings grown into infrastructure), chunking (deceptively simple, endlessly consequential), retrieval pipelines (hybrid search, reranking), semantic search, and memory systems (how the goldfish from Topic 5 gets a past). This is where your Lesson 2 exercise — that 10×10 similarity grid — turns out to have been the seed of a production architecture.