RAG — the open-book exam

RAG (Retrieval-Augmented Generation) answers the question every product hits in week one: how does the model know about MY data? Your docs, your tickets, yesterday's news — none of it is in the weights, and Topic 20 taught you the weight-based path (continued pretraining) needs 100M+ tokens and a GPU budget. RAG takes the other path entirely:

This works because of an asymmetry you've known since Topic 21: models are dramatically more reliable reading from context than recalling from weights. A closed-book exam invites confident fabrication; an open-book exam mostly just requires finding the right page. RAG is the machinery that finds the page.

The architecture — two phases, always:

Picking up right where we left off — here's the RAG architecture, drawn:

And here's the whole loop in code — deliberately minimal, no framework, because you should see there's no magic. It's your Lesson 2 exercise plus a prompt:

from sentence_transformers import SentenceTransformer, util
 
embedder = SentenceTransformer("all-MiniLM-L6-v2")
chunks = [...]                                   # your document pieces (Topic 49)
index  = embedder.encode(chunks)                 # indexing phase, done once
 
def rag_answer(question):
    q = embedder.encode(question)
    top = util.cos_sim(q, index)[0].topk(5)      # retrieve
    context = "\n\n".join(chunks[i] for i in top.indices)
    prompt = f"""Answer using ONLY the context below.
If the answer isn't in the context, say so.        # Topic 21, layer 2
 
Context:
{context}
 
Question: {question}"""
    return llm(prompt)                            # generate

Fifteen lines. Everything else this module teaches is about making each line good: what chunks should be (Topic 49), what replaces that brute-force cos_sim at scale (Topic 50), why encode alone isn't enough (Topic 51), and what belongs between retrieve and generate (Topic 52).

Why RAG won over the obvious alternatives:

• vs. fine-tuning facts in: you know this one cold — Topics 12, 14, 20. Below 100M tokens it barely works, and it trains hallucination.
• vs. "just use the 128K context window": four reasons, all from your own modules. Cost — Topic 5: you pay for the whole whiteboard every turn, and Topic 33: prefill on huge prompts is real quadratic compute; stuffing 100K tokens to answer one question is burning money. Quality — lost-in-the-middle: relevance dilution is real. Freshness — re-index tonight vs. nothing. Access control — the sleeper enterprise killer: retrieval can filter by who's asking ("only chunks this user's role may see"), which no amount of context stuffing or fine-tuning can do.