Embeddings, explained for people who ship things

Strip away the mystique and an embedding is just this: a function that maps a thing — a sentence, a product, a user — to a list of a few hundred numbers, such that similar things land near each other. Nothing else is promised. The numbers mean nothing individually; only distances between vectors mean anything.

The reason embeddings matter is that “similar” is exactly the operation databases are bad at. SQL can find rows where title = 'red running shoes'; it cannot find rows that are about red running shoes. Embeddings turn that fuzzy problem into geometry: embed everything once, embed the query, return the nearest neighbors.

What they’re for (mostly not chatbots)

• Semantic search: retrieve documents by meaning, robust to phrasing. This is also the “R” in RAG, but it’s valuable with no LLM anywhere in sight.
• Deduplication and matching: near-duplicate support tickets, the same product listed twice with different titles, resume-to-job matching.
• Clustering and exploration: embed 100k free-text survey answers, cluster, and read a handful per cluster. Days of tagging work becomes an afternoon.
• Features for boring models: cosine similarity between a user’s embedding and an item’s embedding is often a strong feature in a plain gradient boosting model — no neural serving stack required.

A minimal working example with an open model, using Sentence Transformers:

from sentence_transformers import SentenceTransformer
import numpy as np

model = SentenceTransformer("all-MiniLM-L6-v2")
docs = ["refund not received", "where is my money back",
        "how do I change my password"]
emb = model.encode(docs, normalize_embeddings=True)

query = model.encode(["still waiting for reimbursement"],
                     normalize_embeddings=True)
sims = emb @ query.T          # cosine similarity via dot product
print(np.round(sims.ravel(), 3))   # first two score high, third doesn't

Three lines of setup, and “find tickets like this one” works across phrasings that share almost no words.

The operational details that bite

Normalise, and pick one distance. Cosine similarity and dot product are the same thing on unit-normalised vectors. Mixing normalised and unnormalised vectors, or switching metrics between indexing and querying, produces silently bad rankings — the most common embedding bug in the wild.

Chunking dominates retrieval quality. For documents, what you embed matters more than which model you use. Whole documents blur into vague averages; sentence fragments lose context. Paragraph-ish chunks (a few hundred tokens) with slight overlap are a strong default, and titles prepended to chunks help more than they have any right to.

Never mix vectors from different models. Vector spaces are not compatible across models, or even across versions of the same model. Upgrading the embedding model means re-embedding the entire corpus. Store the model name and version next to every vector; treat “which model made this?” as a schema field, not tribal knowledge.

Exact search first. Under a few hundred thousand vectors, brute-force numpy or a pgvector column in the Postgres you already run is fast enough. Approximate indexes (HNSW and friends) buy speed with recall and operational complexity — adopt them when measured latency forces you to, not because a vendor’s architecture diagram looked impressive.

Evaluate retrieval, not vibes. Collect 50–100 real queries, label which documents should come back, and measure recall@k before and after any change. Embedding pipelines fail quietly; a tiny labelled set is the smoke detector.

The honest summary

Embeddings are a mature, cheap, well-understood tool for one job: making similarity computable. Used for that job — search, matching, dedup, clustering, similarity features — they’re one of the highest-leverage additions to a working data scientist’s toolkit. They don’t require a GPU cluster, a vector-database migration, or a chatbot. Start with a small model, exact search, and a labelled evaluation set, and you’ll be ahead of most production deployments.