Module 1 · Foundations

Lesson 2Inside the Machine

6 topics18 min readTopics 6–11

Lesson 2 keeps the plain language of Lesson 1 but goes further under the hood: actual mechanics, some math intuition, and real numbers. This is the lesson where the "magic" becomes engineering — and where attention, the most beautiful idea in the field, finally gets explained properly.

In this lesson

  1. 6Embeddingsmeaning as coordinatesIn Lesson 1, text became token IDs: [9906, 445, ...]. But an ID is just an index — token 9906 isn't "more" than token 445 in any meaningful…3 min
  2. 7Attentionthe mechanism that made everything possibleI'm teaching attention before transformers, because a transformer is just attention plus plumbing. Understand this and the rest is…4 min
  3. 8Transformersthe full architectureNow assemble the machine. The transformer (from the 2017 paper "Attention Is All You Need") is what happens when you stack attention with a…3 min
  4. 9Parametersthe knobs, quantifiedTime to make "billions of knobs" concrete, because parameter count drives everything you'll do practically: what fits on your Mac M4, what…3 min
  5. 10Training vs Inferencebuilding the brain vs using itTwo completely different modes of a model's existence. Confusing them causes more beginner misconceptions than anything else. Training =…3 min
  6. 11Open-Source vs Closed-Source Modelsthe ecosystem mapLast foundation: the landscape you'll build in. Closed-source (API models): GPT, Claude, Gemini. You send a request over the internet,…4 min

Start with Topic 6: Embeddings, or browse all 90 topics.