Lesson 4Curation, Cleaning, and Truth

Welcome to Lesson 4 — the second half of Module 2. Lesson 3 was about the types of training data; today is about the craft: deciding what deserves to exist in your dataset, scrubbing what's dirty, the one training method that genuinely adds knowledge, and then pulling every thread we've planted into a complete anti-hallucination strategy.

Topic 18: Data Curation — the editor-in-chief job

Curation and cleaning get used interchangeably, but they're different jobs. Curation = deciding the composition: what goes in, in what proportions, at what difficulty. Cleaning = fixing defects in what's already in. Curation is the editor-in-chief choosing which stories run; cleaning is the proofreader fixing typos. Today's topic is the editor's chair.

Decision 1: The mixture. A dataset isn't a pile, it's a recipe with proportions. Suppose you're fine-tuning a support bot: 60% support conversations, 15% structured-output examples (your JSON schemas), 10% edge cases and graceful failures, and — crucially — 15% general instruction data (normal varied chat). That last ingredient isn't filler: it's the standard defense against catastrophic forgetting from Topic 12. Training only on your narrow task pulls the model's knobs so far toward it that general ability decays; mixing general data keeps the knobs anchored. Labs obsess over mixture ratios at pretraining scale (how much code? how much multilingual?) — the same logic applies to your 5,000 examples. Code in the mixture, interestingly, is widely believed to improve reasoning generally, not just coding.

Decision 2: Deduplication. Duplicates are the most damaging invisible flaw in datasets. Three levels:

• Exact duplicates — trivial to catch (hash each example).
• Near-duplicates — same content, trivial differences (whitespace, one synonym). Caught with fingerprinting techniques like MinHash: think of it as a smudged thumbprint that still matches even if the text changed slightly.
• Semantic duplicates — different words, same idea. Caught with embeddings from Lesson 2: embed everything, flag pairs above ~0.9 cosine similarity.

Why care so much? A duplicated example gets its knob-nudges applied twice — you've silently doubled its weight in the recipe without deciding to. At scale, duplication drives memorization (the model regurgitates training text verbatim) and wastes compute. One of the clearest findings from open pretraining efforts (Falcon's RefinedWeb, FineWeb): aggressive deduplication alone measurably improves the resulting model.

Decision 3: Quality filtering — the "better beats more" revolution. The modern approach at scale: train a small, cheap classifier to score every document (FineWeb-Edu famously scored web pages on educational value, keeping only the top slice), and train on the survivors. Result, repeated across many projects: a smaller, well-filtered dataset beats a larger unfiltered one. The Phi models from Lesson 3 are this principle taken to its extreme. At your fine-tuning scale, the equivalent is the LLM-as-judge rubric filter you built in the Topic 17 exercise.

Decision 4: Decontamination — the one beginners never think of. If your training data accidentally contains examples from your evaluation set (or public benchmarks you'll report numbers on), your eval scores become lies — the model isn't solving the test, it memorized it. This is train/test leakage, and it's rampant: benchmark questions litter the internet and sneak into scraped and synthetic data. Standard defense: n-gram overlap checks between training data and every eval you use, remove matches. When you build your own eval set (Module 10), build it first and firewall it from the training pipeline forever.

Decision 5: Difficulty distribution. All-easy examples teach nothing new; all-brutal examples make training unstable and teach the model to flail. You want a gradient — mostly solid mid-difficulty, meaningful hard tail, some easy anchors. This is why Evol-Instruct (Lesson 3) exists: naive generation clusters at "easy."

Topic 19: Dataset Cleaning — scrubbing the dirt

Now the proofreader's job. Real datasets — scraped, synthetic, or human-written — are dirty in predictable ways. Knowing the dirt taxonomy means you can hunt systematically instead of hoping.

The taxonomy of dirt:

1. Structural defects: invalid JSON, missing fields, empty outputs, and — the sneakiest — truncated outputs that end mid-sentence (usually because the generating model hit a token limit). Remember Topic 15: outputs are the tokens your model learns to produce. Train on truncated answers and you literally teach it to stop mid-thou—
2. Artifacts: leftover HTML tags, navigation boilerplate ("Click here to subscribe"), markdown debris, and the infamous assistant-isms. Scraped ChatGPT conversations and lazy synthetic data are riddled with "As an AI language model, I cannot…" — train on those and your model inherits refusals for things it should do, plus every verbal tic ("It's important to note that…", "I hope this helps!"). This is the "slop" inheritance from Topic 17, now with a cleaning procedure: a phrase blacklist.
3. Content defects: factually wrong answers, outdated information, answers that ignore the question. Hardest to catch — requires verification (execution for code, LLM-judge with the source in hand for facts).
4. PII (Personally Identifiable Information): emails, phone numbers, names, addresses that leaked in from real conversations or scraped pages. Legal and ethical hazard — models can memorize and regurgitate it. Regex catches emails/phones; NER (named-entity recognition) tools like Microsoft's Presidio catch names and addresses.
5. Language contamination: stray examples in unexpected languages, or mixed-language answers where they shouldn't be. Cheap to catch with a language-ID library.

The pipeline principle: cheap checks first, expensive checks last. Deterministic validation costs nothing; LLM-judging costs money; human review costs the most. Order accordingly:

import json, re
 
REFUSAL_TICS = ["as an ai language model", "i cannot assist", "i'm just an ai"]
 
def keep(example: dict) -> bool:
    out = example.get("messages", [{}])[-1].get("content", "")
    if not out.strip():                       return False   # empty
    if len(out) < 20 or len(out) > 8000:      return False   # length bounds
    if out.rstrip()[-1] not in ".!?`)\"]}":   return False   # truncation smell
    if any(t in out.lower() for t in REFUSAL_TICS): return False
    if re.search(r"\b[\w.+-]+@[\w-]+\.[a-z]{2,}\b", out): return False  # email PII
    return True
 
data = [json.loads(l) for l in open("raw.jsonl")]
clean = [d for d in data if keep(d)]
print(f"kept {len(clean)}/{len(data)}")      # ALWAYS look at this number

Fifteen lines, and it catches a shocking fraction of real-world dirt. Two habits elevate it to professional grade: always print what you removed (a sample of rejects — your filters might be wrongly killing good examples, e.g. that truncation check would reject perfectly valid answers ending in a code block… which is why you eyeball rejects), and track the kill rate per rule. If one rule removes 40% of your data, either your data has a systemic problem or your rule does — both are worth knowing before training.

Cleaning is a loop, not a phase. You'll clean, train, evaluate, discover a failure pattern ("the model keeps apologizing"), trace it back to dirt you missed ("ah — 200 examples with apology openers"), add a rule, re-clean, retrain. Every mature data pipeline is sediment from cycles like this.

Topic 20: Continued Pretraining — how knowledge actually enters weights

Twice now I've told you "fine-tuning can't reliably add facts" (Topics 12 and 14). Fair question: then how did the facts get in at all? Answer: pretraining — massive-scale next-token prediction on raw text. And that method is available to you too, at smaller scale. It's called continued pretraining (or domain-adaptive pretraining): take a trained model and resume plain next-token prediction on a large corpus of raw domain documents.

Note what's different from everything in Lesson 3: no instruction pairs, no chat template, no roles. Just documents — papers, contracts, codebases, wiki dumps — fed as raw text, exactly like stage 1 of the pipeline diagram. The model isn't learning to answer; it's absorbing the domain's vocabulary, style, and factual regularities into its knobs the same way it originally absorbed the internet.

The threshold that decides everything: scale. Knowledge enters weights through repeated statistical exposure, and that takes volume:

• Below ~50–100 million tokens of domain text → continued pretraining barely moves the needle. Use RAG. (For calibration: 100M tokens ≈ tens of thousands of long documents. Your company wiki is nowhere close.)
• Hundreds of millions to billions of tokens → now it genuinely works. Real examples: Code Llama = Llama 2 + continued pretraining on ~500B tokens of code; BloombergGPT-style finance models trained on enormous financial corpora; medical and legal domain models; and — directly relevant to you — language adaptation: taking an English-heavy model and continued-pretraining on, say, a large Urdu corpus is the standard recipe for building strong models in underserved languages. (With the Lesson 1 caveat: if the tokenizer shreds Urdu into tiny pieces, teams sometimes extend the vocabulary with new tokens and train their embeddings — powerful, but an advanced surgery with real failure modes.)

The price: it breaks the model's manners. You're training on raw documents, so the model drifts back toward being an autocomplete engine — instruction-following and chat behavior erode, because nothing in the corpus reinforces them. Continued pretraining effectively rewinds the model toward base-model behavior, but now domain-smart. Which forces the full recipe — I call it the sandwich:

instruct model (or base)
  → continued pretraining on domain corpus     [knowledge in, manners out]
  → SFT again on instruction data              [manners back]
  → optional preference tuning                 [polish back]

Budget for the whole sandwich, not just the middle. Teams that skip the re-SFT step ship a domain-brilliant model that responds to "summarize this contract" by continuing it.

Forgetting, again, and its standard fix: replay. Trained purely on legal text, the model doesn't just lose manners — it degrades at math, code, and general knowledge. The mitigation is mixing general data back into the domain corpus — commonly 10–50% general text alongside the domain text — plus a gentle learning-rate schedule (a brief warmup ramping the rate up, then a slow decay; sudden full-strength updates on a new distribution shock the knobs).

Where each method sits — the decision table you'll use for years:

Notice the "facts update" row — it's the quiet killer argument for RAG in most products: the world changes, your index updates tonight, while baked-in knowledge goes stale until the next expensive run.

Topic 21: Hallucination Reduction — engineering for truth

The module's capstone, where planted threads pay off. First, the mechanical truth about what hallucination is, because misunderstanding it leads to bad engineering:

The model has no lookup that can fail. When you ask a database for a missing record, it returns null. When you ask a transformer about something it doesn't know, the machinery runs identically to when it does know: attention routes, FFNs fire, out comes a probability distribution, a plausible token gets sampled. There is no built-in "record not found" state. Fluency and confidence are properties of the text style, not signals of knowledge — the model learned confident prose from a trillion tokens of confident prose. Hallucination isn't a bug that malfunctions occasionally; it's the default behavior of a plausibility engine at the edge of its knowledge.

Why it happens — now with receipts from earlier topics:

1. The objective rewards plausibility, not truth (Lesson 1). A fabricated citation in perfect academic format scores beautifully on "plausible next tokens."
2. SFT on unknown facts trains confident guessing (Topic 14) — demonstrated in research: fine-tune a model to answer questions beyond its knowledge and hallucination measurably increases, because the learnable pattern was "always produce a specific answer," not the facts themselves.
3. Preference tuning can make it worse (Topic 16): human and AI judges prefer confident, detailed, longer answers — so stage 3 can accidentally reward the confident-fabrication style over honest hedging.
4. No grounding: asked about your private, recent, or niche facts with nothing in context, plausible invention is all the machinery can do.

A useful word here: calibration. A well-calibrated model's expressed confidence matches its actual accuracy — when it says "I'm fairly sure," it's right most of the time; when it doesn't know, it says so. Raw LLM training does not produce calibration. Engineering has to.

And because no single fix suffices, the professional approach is defense in depth:

Walking the layers with the depth each deserves:

Layer 1 — Training data (this module's contribution). Three moves: (a) never SFT on facts the model can't know — you learned why in Topic 14; (b) include abstention examples — demonstrations where the ideal answer is "I don't know" or "I'd need to check X"; (c) the advanced version, abstention tuning (the R-Tuning idea): probe the model first — ask it a bank of questions, grade its answers, and split them into "knows" and "doesn't know." Then build SFT data where known questions get real answers and unknown ones get honest refusals. You're teaching the model the shape of its own knowledge boundary — which is exactly the calibration raw training never gave it. One warning: overdo the refusal examples and you create the opposite disease, a model that hedges on things it knows perfectly well. Abstention data needs the same mixture discipline as everything in Topic 18.

Layer 2 — Grounding. The single highest-impact layer in practice: don't ask the model to recall — hand it the facts in the context window and instruct it to answer only from what's provided, saying so when the context doesn't contain the answer. Recall from context is dramatically more reliable than recall from weights (the whiteboard beats the memory). This is RAG, getting its full treatment in Module 6; for now, register it as anti-hallucination measure #1 in nearly every serious product.

Layer 3 — Inference settings. Cheap wins: for factual tasks, drop the temperature (Lesson 1) — you want the model's top-probability belief, not adventurous sampling. And a genuinely elegant detection trick built on something you already understand: consistency checking. Ask the same question 5 times at moderate temperature. If the model knows, the answers agree — the probability mass is concentrated. If it's fabricating, the answers scatter (three different founding years, two different names) because it's sampling from a flat, uncertain distribution. Disagreement is a measurable uncertainty signal — the research version is called semantic entropy, but the intuition is just "a liar's stories don't stay straight." Costs 5× inference, worth it for high-stakes paths.

Layer 4 — Verification. Check outputs after generation: claims cross-checked against the retrieved sources ("is every statement in this answer actually supported by the provided context?" — an LLM-judge task with the receipts in hand, much easier than judging truth in a vacuum); structured outputs validated against schemas; code executed; citations resolved to confirm they exist. Anything verifiable should be verified — Topic 17's golden rule, resurfacing.

Layer 5 — Product design. The layer engineers forget because it isn't ML: show citations so users can check (and so fabrications become visible); route high-stakes outputs (medical, legal, financial) through human review; design UI language that sets accurate expectations. A product that displays its sources partially contains the hallucinations that slip through layers 1–4.

And the honest closing note, which distinguishes engineers from hype-merchants: hallucination is reduced, never eliminated. It's inherent to the plausibility-engine mechanism — every layer lowers the rate; none reaches zero. The professional question is never "how do I make it stop hallucinating?" but "what hallucination rate can this product tolerate, and which layers get me there?" A brainstorming tool tolerates a lot. A drug-dosage assistant tolerates almost none — hence layer 5's human in the loop.

Module 2: complete. The full data discipline is yours: demonstration, preference, and synthetic data; formatting that doesn't silently kill runs; curation with mixtures, dedup, and decontamination; cleaning as a loop; continued pretraining as the true knowledge path; and truth as an engineered, layered property.

Next: Module 3 — Fine-Tuning Methods. LoRA and QLoRA (where Lesson 2's 8× training-memory flag finally cashes out — you'll see how to fine-tune a 7B model on hardware like your own Mac), DPO and RLHF (the algorithms that consume your Topic 16 preference pairs), quantization, checkpoints, adapters, and GGUF. This is the module where you stop preparing data and start running training.