What it can't do

Now that you know how it works, its weak spots make sense. Here are the three that catch people out. (Forgetting isn't on the list. You saw in lesson 6 that it's the interface, not a flaw.)

It has a cutoff. The weights encode text up to the training cutoff. Anthropic lists Claude Haiku 5.5 with a reliable knowledge cutoff of June 2026 (checked October 2026). Anything after that date reaches the model only through the context window: pasted text, retrieval or a tool call. Design rule: if the answer depends on anything recent, volatile or private, supply it; never rely on the weights. The sandbox here has no tools, so this is the raw model.

It hallucinates. Lesson 4's compression is the mechanism. Where the corpus was thin, the next-token distribution over the specifics is flat, but decoding still emits a token, and the result is grammatical and confident. Fluency is not evidence. Illustrative odds for a book that doesn't exist:

Every draw above was near a coin flip; the output reads as certain. That is a hallucination, and nothing in the token stream marks it.

Post-training leans against it. Anthropic's interpretability work on an earlier Claude found a default “can't answer” circuit that a “known entity” feature switches off; the failure mode is a name that trips the familiarity feature without any facts attached. Expect the worst behaviour on half-known entities: obscure papers, minor libraries, people with common names, plausible-looking URLs and version numbers.

Arithmetic and counting are approximate. The model has no arithmetic unit; it learned numeric patterns from text. The same study caught Claude adding two numbers with parallel heuristics (an approximate magnitude path and an exact last-digit path) and then, asked to explain, describing the schoolbook carry it never used. So its account of its own method is not a trace. Anything exact (sums, counts, dates, string lengths, sorting) belongs in a tool or in code; the Give your model tools course wires that up.

Anthropic's own mitigation list, in order of leverage: give the model explicit permission to say it doesn't know; ground it in supplied text and ask for direct quotes before claims; have it verify or cite; sample several times and compare, since a fabricated answer tends not to repeat. And treat run-to-run variance as a signal, not noise.

Your turn. Ask about something obscure or made up (a half-familiar book works well), and give Claude permission to say it doesn't know.