Module 2 · Datasets & Training

Lesson 4Curation, Cleaning, and Truth

4 topics16 min readTopics 18–21

Welcome to Lesson 4 — the second half of Module 2. Lesson 3 was about the types of training data; today is about the craft: deciding what deserves to exist in your dataset, scrubbing what's dirty, the one training method that genuinely adds knowledge, and then pulling every thread we've planted into a complete anti-hallucination strategy.

In this lesson

  1. 18Data Curationthe editor-in-chief jobCuration and cleaning get used interchangeably, but they're different jobs. Curation = deciding the composition: what goes in, in what…4 min
  2. 19Dataset Cleaningscrubbing the dirtNow the proofreader's job. Real datasets — scraped, synthetic, or human-written — are dirty in predictable ways. Knowing the dirt taxonomy…3 min
  3. 20Continued Pretraininghow knowledge actually enters weightsTwice now I've told you "fine-tuning can't reliably add facts" (Topics 12 and 14). Fair question: then how did the facts get in at all?…4 min
  4. 21Hallucination Reductionengineering for truthThe module's capstone, where planted threads pay off. First, the mechanical truth about what hallucination is, because misunderstanding it…6 min

Start with Topic 18: Data Curation, or browse all 90 topics.