Skip to content
bits
Sign inStart free
Retrieval without tears · Lesson 3 of 9

Embeddings and similarity

Tutor:

“How do I get my money back?” and “Refund policy” share no words, yet they're about the same thing. Search by meaning needs a way to see that.

An embedding model turns a piece of text into a list of numbers, called a vector. It's trained so that texts with similar meaning get similar vectors. You embed every chunk once, ahead of time. At question time you embed the question and look for the chunks whose vectors sit closest to it.

“Closest” is usually cosine similarity: how closely two vectors point the same way. 1 means the same direction, 0 means unrelated.

Tutor:

Here's a toy you can run. Real vectors have about a thousand numbers; these have three, made up by hand so the idea is easy to see. Think of the three numbers as “about money”, “about delivery” and “about pets”.

// Hand-made 3-number "embeddings", for illustration only.
const chunks: Record<string, number[]> = {
"Refund policy": [0.9, 0.1, 0.0],
"Shipping times": [0.1, 0.9, 0.1],
"Office dog rules": [0.0, 0.2, 0.9],
};
const question = [0.8, 0.3, 0.1]; // "How do I get my money back?"
function cosine(a: number[], b: number[]) {
let dot = 0, lenA = 0, lenB = 0;
for (let i = 0; i < a.length; i++) {
dot += a[i]! * b[i]!; lenA += a[i]! ** 2; lenB += b[i]! ** 2;
}
return dot / Math.sqrt(lenA * lenB);
}
for (const [title, vector] of Object.entries(chunks)) {
console.log(cosine(question, vector).toFixed(2), title);
}
TypeScriptUTF-8● lines 9–15: cosine similarity
OUTPUT
npx tsx toy.ts
0.96 Refund policy
0.46 Shipping times
0.19 Office dog rules
Tutor:

The refund chunk wins without sharing a single word with the question. In a real model, nobody chooses what the numbers mean. The model learns them, and they don't map to neat labels like “money”.

quick check+10 neurons
Tutor:

A chunk scores 0.96 against the question. What does that tell you?

    Type your answer or tap an option · Enter checks

    Tutor:

    Which model? Anthropic doesn't make an embedding model. Its docs point to Voyage AI (now part of MongoDB). As of October 2026 the general-purpose family is voyage-4-large (best quality), voyage-4 (balanced) and voyage-4-lite (fastest and cheapest). They return 1,024 numbers by default. Shop around if you like; Anthropic's docs say to compare vendors on your own data.

    Two rules for real use. Tell the model whether it's embedding a query or a document (input_type), because it embeds them slightly differently to make questions match answers. And use one model family for everything: vectors from different families live in different spaces, so comparing them gives nonsense. (The voyage-4 models share one space, which is unusual.)

    // Voyage's Embedding API, via MongoDB Atlas. Needs VOYAGE_API_KEY.
    export async function embed(texts: string[], inputType: "query" | "document") {
    const res = await fetch("https://ai.mongodb.com/v1/embeddings", {
    method: "POST",
    headers: {
    "content-type": "application/json",
    authorization: `Bearer ${process.env.VOYAGE_API_KEY}`,
    },
    body: JSON.stringify({ input: texts, model: "voyage-4", input_type: inputType }),
    });
    if (!res.ok) throw new Error(`Voyage: ${res.status} ${await res.text()}`);
    const json = (await res.json()) as { data: { embedding: number[] }[] };
    return json.data.map((d) => d.embedding); // one 1,024-number vector per text
    }
    TypeScriptUTF-8
    Tutor:

    Voyage's vectors come out with length 1, so a plain dot product gives the same answer as cosine similarity, a little faster.

    quick check+20 neurons
    Tutor:

    Your index was built with voyage-3.5. You switch the query side to a newer model from a different family, with the same 1,024 numbers. What happens?

      Type your answer or tap an option · Enter checks

      Down the rabbit hole