Adapter Tuning & PEFT — the "don't touch the walls" idea

The family name for everything in this lesson is PEFT: Parameter-Efficient Fine-Tuning. One shared insight powers the whole family:

Freeze 7B params (14 GB, sitting quietly) and train, say, 40M new ones (40M × 16 bytes ≈ 0.6 GB of training overhead). The 112 GB monster collapses to under 20 GB. The whole trick is choosing what small thing to train so that it can meaningfully steer a frozen giant.

Daily-life analogy: Renovating a rented apartment. You can't demolish walls (the base weights are frozen — you don't own them, and touching everything is what made it expensive). But you can add furniture, lamps, and removable fixtures — small additions that completely change how the space works, and can be swapped or removed without a trace.

The family members, in historical order:

• Classic adapters (2019): insert tiny bottleneck layers between the transformer's existing layers — little "squeeze down to 64 dims, process, expand back" modules. Worked, but the inserted layers sit in the inference path forever: permanent extra latency. Furniture you can never remove from the hallway.
• Prompt tuning / prefix tuning: don't add layers at all — learn a handful of virtual tokens (trainable embedding vectors, not real words) that get prepended to every input, steering the frozen model like a magic incantation. Elegant, ultra-cheap, but weaker steering power and it eats context window.
• LoRA (2021): the one that won, so decisively that "fine-tuning" in open-source practice now means LoRA by default. Why it won is the next topic — the short version: all the steering power of adapters, with zero inference latency after a merge step.
• Modern refinements you'll see in tooling: DoRA (splits weight updates into magnitude and direction components; often edges out LoRA at the same size), rsLoRA (a scaling fix for high ranks). Know the names; reach for them when squeezing the last few percent.

One product-level consequence worth planting now: because the base stays untouched and adapters are tiny files, you can keep one base model in memory and hot-swap many adapters — a support-tone adapter, a JSON-extraction adapter, a per-customer adapter — like outfits on one mannequin. Serving frameworks exploit this (multi-LoRA serving in vLLM and friends, Module 5), and it's the economic foundation of "a fine-tune per customer" products. Full fine-tuning could never afford that: each customer would need their own 14 GB copy.

---