Last post we made peace with a hard truth: the context window only fits so much, so we have to choose what goes in. Choosing needs a way to ask "which text is relevant to this question?" And computers cannot compare meaning. They can only compare numbers. So the whole game becomes: turn meaning into numbers. Those numbers are vectors.
Ctrl+F(finding/searching) is not understanding
Say our Blog app has fifty posts and a reader asks "which post covers shipping containers to production?" A keyword search for "shipping containers" finds nothing, because the post they want says "deploying with Docker." Same meaning, zero shared words. Keyword search matches spelling. We need something that matches meaning.
Meaning as coordinates
Here is the trick, in a deliberately tiny version. Pick two qualities and score every word on them by hand, say how techy it is and how foody it is:
# each word becomes [techy, foody]words = { "fastapi": [0.9, 0.1], "docker": [0.8, 0.0], "biryani": [0.1, 0.9], "pasta": [0.0, 0.8], "recipe api": [0.7, 0.6],}Each word is now a point on a 2D map, and something interesting already happened:
"fastapi" and "docker" sit in the same corner. So do "biryani" and "pasta". We never told the computer these pairs are related, their coordinates just landed close together.
"recipe api" floats between the two corners, because it genuinely belongs to both worlds.
Related things are near each other. That one sentence is the entire foundation of semantic search.
A list of numbers like [0.8, 0.0] is a vector. Nothing scarier than that.
From 2 axes to 1536
Our map has an obvious problem: on just techy and foody, "docker" and "fastapi" look identical, and sarcasm, tense, and topic do not fit anywhere. Two axes cannot hold much meaning.
Real models fix this with scale. Instead of 2 hand-picked axes, they use hundreds or thousands of learned ones, and place text on that map automatically. Feed one a sentence and you get back something like:
[0.0123, -0.0311, 0.0045, ..., 0.0198]That is 1536 numbers for one sentence with a typical model. Nobody labels these axes and nobody can tell you what axis 812 means. The model learned, from mountains of text, whatever dimensions it needed so that related text ends up close. A vector produced this way is called an embedding, a word you will now see in every GenAI job posting. 🗺️
Back to our Blog
You can probably see the plan forming. Store an embedding next to every blog post. When a question comes in, embed the question too, and the answer to "which post is relevant?" becomes "which stored vector is nearest to the question's vector?" "Deploying with Docker" and "shipping containers to production" land close on the map, even with zero words in common. This is the trick behind every "chat with your docs" app you have seen.
We are not coding it yet, and honestly we cannot: two pieces are missing. Where do real embeddings come from, and what does "nearest" even mean when the map has 1536 dimensions? You cannot eyeball that. There is a neat bit of math for it with a name that sounds fancier than it is: cosine similarity. Both pieces are next. 🙂