Skip to content
bits
Sign in
Give your model tools · Lesson 7 of 9 · 25 min

Strict inputs, honest failures

Desk: best at a computer, with a terminal
Desk: best at a computer, with a terminal

Verified ai 7.0.162zod 4.6.7agents 0.33.0

Start from b-06 on GitHub. Starting here, or behind? Get the code at that tag:

Shell
git clone https://github.com/ryanyogan/klippy
cd klippy
git checkout b-06
Tutor:

Tools go wrong in two ways. The model sends bad input, like a time zone that doesn't exist. Or the input is fine and the call fails: the weather service is down. In both cases the worst thing Klippy can do is crash, or make something up. The right thing is to tell the model what went wrong, so it can tell the user. In this lesson you'll make both failures honest.

Tutor:

1. Input the schema can't describe

getTime takes a timeZone string. Ask for the time in "Mars/Olympus_Mons" and that's a string, so the schema lets it through, and toLocaleString throws. A schema can say "a string"; it can't say "a real time zone". So add a check that runs in your code. In apps/agent/src/tools/time.ts, replace the schema:

inputSchema: z.object({
timeZone: z
.string()
.describe('An IANA time zone, e.g. "Europe/Berlin" or "America/New_York"')
// The schema Claude sees says "a string". This check runs in your code, after Claude calls.
.refine(isTimeZone, 'Not an IANA time zone. Use a name like "Europe/Berlin".'),
}),
TypeScriptUTF-8at b-07● lines 2–6: apps/agent/src/tools/time.ts
Tutor:

and add the check at the bottom of the file:

function isTimeZone(timeZone: string) {
try {
new Intl.DateTimeFormat('en-GB', { timeZone });
return true;
} catch {
return false; // Intl throws a RangeError for a zone it doesn't know
}
}
TypeScriptUTF-8at b-07● lines 1–8: apps/agent/src/tools/time.ts
Tutor:

  • .refine(isTimeZone, message) runs isTimeZone on the value after the model calls. If it returns false, the input is rejected and execute never runs.

  • Intl.DateTimeFormat throws for a zone it doesn't know, which makes it a handy test.

  • The message is written for the model: what was wrong and what to send instead ("Use a name like Europe/Berlin"). A good error is a hint.

The AI SDK doesn't stop when input is rejected. It sends the error back to the model as the tool's result (Claude's API marks it is_error), and the model gets its next step to fix the call or explain.

Tutor:

2. A call that fails

Right now, if Open-Meteo answers with an error page, getJson tries to read it as JSON and Klippy fails in a confusing way. Check the status first. In apps/agent/src/tools/weather.ts:

async function getJson(url: string) {
const response = await fetch(url);
// A failed call is a result too: throw, and Claude is told the tool failed, so it can say so honestly.
if (!response.ok) throw new Error(`The weather service is not answering (HTTP ${response.status}). Try again later.`);
return response.json<any>();
}
TypeScriptUTF-8at b-07● lines 3–4: apps/agent/src/tools/weather.ts
Tutor:

response.ok is false for any 4xx or 5xx status. Throwing here is right: the AI SDK catches it, and the model is told the tool failed, with your message. It can then say "the weather service isn't answering" instead of inventing a forecast.

Tutor:

3. See the errors in your log

Add one field to the step log in apps/agent/src/ask.ts, after toolResults:

toolResults: step.toolResults.map((result) => ({ tool: result.toolName, output: result.output })),
// A failed tool call (bad input, or a service that's down) goes back to Claude as an error.
toolErrors: step.content.flatMap((part) => (part.type === 'tool-error' ? [{ tool: part.toolName, error: String(part.error) }] : [])),
TypeScriptUTF-8at b-07● lines 2–3: apps/agent/src/ask.ts
Tutor:

A failed call doesn't appear in toolResults; it's a tool-error part of the step. This pulls them out, one per failed tool.

Now ask for the time on Mars. Our run:

recorded output
pnpm dev
{"step":0,"finish":"tool-calls","toolCalls":[{"tool":"getTime","input":{"timeZone":"Mars/Olympus_Mons"}}],"toolResults":[],"toolErrors":[{"tool":"getTime","error":"AI_InvalidToolInputError: Invalid input for tool getTime: AI_TypeValidationError: Type validation failed: Value: {\"timeZone\":\"Mars/Olympus_Mons\"}.\nError message: [\n {\n \"code\": \"custom\",\n \"path\": [\n \"timeZone\"\n ],\n \"message\": \"Not an IANA time zone. Use a name like \\\"Europe/Berlin\\\".\"\n }\n]"}],"text":"","tokens":645}
{"step":1,"finish":"stop","toolCalls":[],"toolResults":[],"toolErrors":[],"text":"The time zone \"Mars/Olympus_Mons\" isn't a valid IANA time zone that my tool recognizes. I can only provide times for authoritative timezone databases like those for Earth. For Mars, you'd need a specialized Mars timekeeping system not available in this tool.","tokens":817}
Tutor:

Step 0: the model sent the bad zone, your refinement refused it, and the error went back. Step 1: the model read the error and told the truth. (The free fallback model again; Claude does the same with the same error.)

Klippy's test/failures.spec.ts checks both: a bad zone never reaches execute, and a 502 from the weather service comes back to the model as an error. Ours: 27 passed.

Tutor:

Write an error Claude can use

The sandbox can't run your tools, but it can show what a good error does. Write the error your weather tool would return for a misspelled city, and ask Claude what it does next.

Your turn

Goal Write the error your weather tool returns for a misspelled city, so well that Claude's next step is obvious and right

What good looks likeThe coach checks the reply
  • Says it's a tool errornot checked yet
  • Asks what Claude would donot checked yet
  • Ran at 1.0not checked yet
  • Answers in a sentence or twonot checked yet
  • The error says what went wrong and what to try insteadnot checked yet
  • Claude's next step follows from your error's suggestion, not from a guessnot checked yet
  • Claude doesn't make up the weathernice to have

Run your prompt and the replies land here.

    ~0 tokens

    Turn on JavaScript to run prompts.

    Tutor:

    What about strict?

    Claude also offers strict tool use: mark a tool strict: true and Claude's input is guaranteed to match your schema: the right types, every required field, only allowed enum values. The AI SDK has a strict option per tool for it.

    Two honest notes. First, strict guards the shape, not the meaning: a strict timeZone is still a string, and "Mars/Olympus_Mons" still fits. Strict schemas also can't use limits like a maximum length, so checks like .max(1000) on a note and .refine on a zone stay in your code either way. Second, Klippy doesn't set strict, because the version of the Agents SDK it uses (0.33.0) doesn't pass that flag on to the model. A flag that does nothing would mislead you. Your code's own checks are what Klippy relies on, and they work with any model.

    quick check+20 neurons
    Tutor:

    What does strict tool use guarantee?

      Pick one option

      Pick one, then Check.

      Down the rabbit hole