Tutor:
Here's the one idea in this lesson. Pretraining is where the model gets what it knows. Post-training is where it learns how to behave. Everything below is those two sentences, slowed down.
Tutor:
A base model is a document continuer. Given a question with no framing, the highest-likelihood continuation is often more of the page it looks like it came from: a forum thread, a quiz sheet, a list of FAQs. Same prompt, base model against assistant:
Base model vs Assistantillustrative: made-up numbers that show the idea, not output from a real modelcompare
promptHow do I boil an egg?
How do I boil an egg? I tried twice and the yolk went grey both times. Posted in Kitchen Help · 14 replies · Sort by: newest
Lower the egg into boiling water. Cook 6–7 minutes for a jammy yolk or 10–12 for hard, then cool it in cold water before peeling.
Tutor:
Both are good next-token guesses. Only one is useful to you. Turning the left into the right is called post-training.
Tutor:
Stage one is supervised fine-tuning on conversations: the same next-token objective, but the corpus is now dialogues in a fixed format (a system note, user turns, assistant turns), so the model learns to continue the transcript as the assistant. The chat format you'll send next lesson is that training format, which is why the roles carry weight.
Stage two is preference optimisation. The model samples several responses per prompt; a reward signal ranks them; reinforcement learning shifts the policy towards what ranks well. The glossary's RLHF is the human version: people rank two or more candidates. Anthropic's Constitutional AI adds self-critique and revision against written principles and then RL from AI feedback, so most of the ranking is done by a model judging against the constitution. The Transparency Hub describes Claude Haiku 5.5 as having had substantial post-training toward those values. Post-training uses far less text than pretraining; it steers behaviour more than it adds knowledge.
Tutor:
Current Claude models also do adaptive thinking: per request, the model decides whether to reason before answering and how much, with an effort level you can set. The reasoning is ordinary sampled tokens, counted against max_tokens and billed as output. The sandbox here runs with thinking off and low effort, which is fine for one-line answers and wrong for anything multistep.
The invariant: it is still next-token prediction, now over the distribution of what a helpful, honest assistant would say. Knowledge mostly comes from pretraining; behaviour mostly comes from post-training.