Dataset Formatting — where fine-tunes silently die

The least glamorous topic in this module, and the cause of the majority of "my fine-tuned model is broken" posts. Attend closely.

Storage format: the universal standard is JSONL — one JSON object per line, no commas between lines, no wrapping array. Streams efficiently, appends trivially, and every training library expects it.

The chat template — the concept that actually matters. The model never sees your neat messages array. Before training or inference, the array gets flattened into one token stream using the model family's chat template — special tokens marking who's speaking. Qwen and many others use ChatML:

<|im_start|>system
You are a support agent.<|im_end|>
<|im_start|>user
Why is my API key rejected?<|im_end|>
<|im_start|>assistant
Usually one of three causes: ...<|im_end|>

Llama 3 uses entirely different markers (<|start_header_id|>user<|end_header_id|> …). These marker tokens are how the model knows a turn ended and whose turn it is. Which sets up the classic silent killer: train with one template, serve with another (or with none), and the model receives token patterns it never saw in training. Output degrades into rambling, format-breaking, never stopping — and nothing errors out. It just quietly produces garbage. When someone's fine-tune "doesn't work," template mismatch is suspect #1.

The professional habit: never hand-write templates. Let the tokenizer apply its own:

from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct")
 
text = tok.apply_chat_template(example["messages"], tokenize=False)
# ALWAYS print one example and read it with your eyes before training:
print(text)

That print is not optional ceremony. Inspecting one fully-rendered example catches template bugs, missing system prompts, and the next issue — before you burn GPU hours.

Two more formatting killers:

1. The EOS token. The end-of-sequence token (<|im_end|>, etc.) after the assistant's answer is how the model learns to stop talking. If your formatting pipeline drops it, you train a model that never learned that answers end — it generates until it hits the token limit, every time. A one-token bug with a very memorable symptom.

2. Loss masking. Recall from Lesson 1: training = predict next token, score the miss, nudge. But your training rows contain both the user's tokens and the assistant's tokens. Should the model be nudged toward producing the user's words? No — you'd be training it to imitate users (models that randomly start asking themselves questions — that's this bug). The fix is masking: loss is computed only on assistant tokens; prompt tokens are read for context but generate no learning signal. Visually:

Good news: modern training libraries (TRL, which we'll meet in Module 5) handle masking for you — assistant_only_loss style settings — but you must know it exists, because when it's misconfigured, the symptoms are subtle quality rot rather than a crash.

Last rule: train the way you'll serve. Same chat template, same system prompt (or same distribution of system prompts), same input structure. Every train/serve mismatch is a small tax on quality; template mismatch is a total one.