Semantic Search — what embeddings miss, and the hybrid fix

Semantic search — embed query, find nearest chunks — has been our quiet workhorse since Lesson 2. Time to be honest about where it fails, because production quality lives in the failure modes:

1. Exact-identifier blindness. Embeddings match meaning, and identifiers barely have any. Search "error E-4032" and the nearest neighbors include E-4031 and E-4088 — semantically they're all just "error codes." Product SKUs, function names, ticket numbers, person names: the fuzzy-matching superpower becomes a liability precisely when the user knows exactly what they want.
2. Negation and numbers — Lesson 2's warning, still true: "flights under $200" and "flights over $200" embed uncomfortably close.
3. Question–answer asymmetry. "How do I reset my password?" and the doc paragraph that answers it share little surface form. Good retrieval embedders are trained on Q→A pairs to bridge this — and here's the practical gotcha that burns everyone once: many such models require role prefixes ("query: ..." vs "passage: ...") because they embed questions and documents into deliberately different regions. Omit the prefix and quality silently craters. Read your embedder's model card (Topic 43's skill, paying off).

Now meet the 50-year-old rival that happens to be strong exactly where embeddings are weak: BM25, the classic keyword algorithm behind traditional search engines. It scores documents by exact term matches, weighted so that rare words count heavily (matching "E-4032" is a jackpot; matching "the" is nothing) and diminishing returns apply to repetition. No neural anything — and it nails identifiers, names, and exact phrases every time they appear.

Two complementary specialists suggest the obvious architecture, and it is the production default — hybrid search: run both searches, then fuse the rankings. The standard fusion is RRF (Reciprocal Rank Fusion), almost embarrassingly simple: each document scores Σ 1/(60 + rank) across the lists it appears in. No score normalization, no tuning, robust — a document ranked well by either specialist surfaces, and one ranked well by both dominates. Every serious stack (including pgvector setups — Postgres has full-text search built in) supports this pattern.

Last practical layer — choosing the embedder: the MTEB leaderboard is the standard comparison (with the usual Topic-18 caveat about benchmark contamination); dimensions trade quality against storage/speed (your Topic 50 RAM math); Matryoshka embeddings are a lovely modern trick — trained so the first N dimensions form a valid smaller embedding, letting you truncate 1536→256 dims for a cheap first pass and rescore with full vectors. And the iron rule stands, third and final repetition: one embedding model per index. Changing embedders means re-embedding everything — a real migration cost to plan for, not discover.

---