Filtered and Hybrid Search — Metadata, BM25, and Broken Recall
Real queries are nearest vectors for this tenant, in stock, that this user may see, and often matching an exact error code. Filters fight ANN structures: post-filtering returns too few hits, a matching-only walk strands, and a filter-aware walk gets expensive when the predicate is selective. Hybrid search adds a BM25 leg and fuses ranks so one score scale cannot silently win.
- 1Gist
- 2Maps
- 3Q&A
- 4Sandbox
Voice readout needs Web Speech Synthesis in this browser.
Question ladder
L1
A filtered query returns 3 hits when k is 10. Why?
Answer
Post-filtering. The engine took an unfiltered top-k and then dropped non-matches. Pass the filter into the kNN clause, raise the candidate budget, or fall back to an exact scan of the matching ids.
L2
Why does a strict pre-filter break HNSW?
Answer
The graph was built over all nodes. If you may only walk matching nodes, that piece of the graph is often disconnected, so the walk strands near the entry point.
L3
What does filter-aware traversal do instead?
Answer
It walks through non-matching nodes and only admits matches into the result set. Recall holds. Cost grows as the filter tightens, because the walk visits many non-matches before it fills k.
L4
When is the exact scan the right plan?
Answer
When selectivity is low enough that the matching ids are cheaper to score than the graph walk. Engines estimate cardinality and switch under a threshold. At 2 percent in the sandbox, exact touches about 25 ids.
L5
RRF or a weighted sum?
Answer
Reciprocal rank fusion when you have no labels or the legs have incomparable scales. A weighted sum after normalization when labels can tune the weight and a very confident BM25 hit should count.
L6
How do you isolate 10,000 tenants, a few of them huge?
Answer
Huge tenants get their own index or shard. Small tenants share a partition and hit an exact scan of their ids. The tenant predicate is enforced on the server. A missing filter is a data leak, not a relevance bug.
L7
What is ACORN?
Answer
A filtered HNSW approach that builds a denser graph and, during search, expands two hops through non-matching nodes so the matching nodes stay navigable. Weaviate ships a strategy in that family.
Failure modes
Post-filter returns a short list
You asked for 10 and got 3. The filter ran after the ANN cutoff. Oversample, or push the predicate into the walk.
Matching-only walk
At 10 percent selectivity the sandbox strands after about one node and recall collapses. That is the bug you build by deleting edges.
Raw score blend
BM25 runs from 0 to 20. Cosine runs from 0 to 1. A 0.5 and 0.5 blend replays the BM25 order.
Anti-correlated filter
Winter coats inside swimwear. The matches sit far from where the walk starts, so even a correct filter-aware search explores a huge region.
Misconceptions
A metadata filter is a WHERE clause the index ignores until the end.
On a graph, when you apply the predicate changes recall, result count, and cost. Selectivity picks the strategy per query.
One shared index plus a tenant id is isolation.
It is isolation only if the filter cannot be skipped. Tiny tenants should not pay for an ANN walk of everyone else's vectors.
Hybrid means you add the two scores.
Adding them lets the wider numeric range win. Fuse by rank, or normalize first.
Interviewer traps
Re-teaching the RAG query path, chunk windows, and the cross-encoder stack.
Point at the hybrid retrieval page for that path. Stay on selectivity, the five strategies, and score scales.
Explaining BM25's k1 derivation from scratch.
Name saturation and length normalization in one sentence, then show why the BM25 range swamps cosine.
Design scenario
Same prompt for every reader.
Requirements
k = 10 whenever 10 matching documents exist. No cross-tenant hits. Exact codes must not depend on the embedding.
Traffic / scale
10,000 tenants, a few huge, many tiny. Filters range from half the corpus down to about 2 percent.
Latency
A 2 percent filter should not walk the whole graph. A paraphrase query may pay for two legs.
Consistency
The tenant predicate is mandatory on every request. Fusion does not compare raw scores across legs.
Availability
A failed lexical leg must be visible. Silent fallback to vectors-only hides SKU misses.
Failure assumptions
- A client sends the filter as a post-filter.
- A tiny tenant is routed into the shared HNSW.
- BM25 and cosine are added without normalization.
Constraints
- Pick the strategy from selectivity, per query.
- Do not design a second embedding model on this whiteboard.
Prompt
Search a multi-tenant catalog. Queries combine a paraphrase, an exact SKU, and filters for tenant, stock, and visibility.
API
Which parameter carries the filter into kNN, and which one is too late?
Data
What does each leg return before fusion, and what is the fusion key?
Architecture
Where do huge tenants live, and where do small tenants get an exact scan?
Pick a filter plan, then fuse
The right graph strategy is a function of selectivity. Fusion is a second decision, about score scales.
- 1
Estimate selectivity
A tenant, a stock bit, or an ACL either leaves most rows or a thin slice. The estimate is per query. - 2
Choose the walk
Post-filter when most rows pass. Filter-aware traversal in the middle. Exact scan when few ids match. - 3
Run both legs
Filtered BM25 catches exact tokens. Filtered kNN catches paraphrase. Each returns its own ranked list. - 4
Fuse by rank
Reciprocal rank fusion ignores the fact that BM25 spans a wider range than cosine. A raw sum does not.
Overview
Real queries are never only "nearest vectors." They are nearest vectors for this tenant, in stock, published after March, that this user may see, and often also matching the exact error code they typed.
Filters fight ANN structures. Post-filtering returns too few results. A walk that refuses to touch non-matching nodes strands. Filter-aware traversal stays correct and gets slow when the filter is selective. Hybrid search adds a lexical BM25 leg and fuses the two rankings.
This page goes deeper on those index mechanics than Hybrid Retrieval — BM25 + Vectors, Reranking & Metadata Filters, which stays the reference for the RAG query path. BM25's formula, as a lexical scoring model, stays on Query DSL, Relevance Scoring (TF-IDF/BM25) & Filters vs Queries. The inverted index under the lexical leg is Inverted Index, Analyzers, Tokenization & Mappings.
The filtering strategies
Decisions
- 1
1. Query vector plus filter
- next2. How selective?
- ?
2. How selective?
- most pass3a. Post-filter, oversample
- tighter3. Medium or tiny?
- 3
3a. Post-filter, oversample
- next4. Top k
- ?
3. Medium or tiny?
- medium3b. Filter-aware walk
- tiny3c. Exact scan of matches
- 5
3b. Filter-aware walk
- next4. Top k
- 6
3c. Exact scan of matches
- next4. Top k
- 7
4. Top k
- next5. Fewer than k hits?
- ?
5. Fewer than k hits?
- yes6. Oversample or go exact
- no6. Return the hits
- 9
6. Oversample or go exact
- 10
6. Return the hits
Lesson map
Filtered and Hybrid Search — Metadata, BM25, and Broken Recall
Real queries are nearest vectors for this tenant, in stock, that this user may see, and often matching an exact error code. Filters fight ANN structures: post-filtering returns too few hits, a matching-only walk strands, and a filter-aware walk gets expensive when the predicate is selective. Hybrid search adds a BM25 leg and fuses ranks so one score scale cannot silently win.
Architecture. Architecture
Select a node to see why it exists, or an edge to see the protocol, direction, effect, and consequence.
Mermaid export
flowchart TB q["1. Query vector plus filter"] sel["2. How selective?"] post["3a. Post-filter,"] mid["3. Medium or tiny?"] q -->|1. Query vector plus filter| sel sel -->|most pass| post sel -->|tighter| mid
| Strategy | How | Recall | Result count | Cost | Who does it |
|---|---|---|---|---|---|
| Post-filter | ANN top k, then drop non-matches | Drops as selectivity falls | Often fewer than k | Low | Naive apps, older plugins |
| Post-filter plus oversample | Ask for about k divided by selectivity, then filter | Good | k | Grows as 1 / selectivity | Many app-side implementations |
| Filter-aware traversal | Walk the full graph, admit only matching nodes | Good | k | Grows as the filter tightens | Lucene and Elasticsearch filtered kNN, hnswlib's filter, OpenSearch |
| Matching-only walk | Walk only matching nodes | Collapses when matches are not connected | Short | Low until it strands | The bug you build by accident |
| Exact on matching ids | Brute force over filtered ids | 1.0 | k | selectivity times N | Lucene and Qdrant, automatically, below a threshold |
| Payload-aware graph | Extra links inside filter groups, or two-hop expansion | Good | k | Moderate | Qdrant payload links, Weaviate ACORN |
What the simulation shows
At 50 percent selectivity every strategy returns about 10 hits. The matching-only walk is the cheapest here, about 126 distance evaluations, because the matches are still connected. Plain post-filter is next, about 230 evaluations, with recall about 0.99.
At 10 percent, plain post-filter returns about 4 results on average and recall is about 0.40. A matching-only walk strands after about one node. Filter-aware traversal keeps recall at 1.0 and evaluates about two-thirds of the corpus.
At 2 percent, an exact scan over the roughly 25 matching ids is both exact and about 60 times cheaper than the graph walk. That is why Lucene, Qdrant, and others estimate cardinality first and switch to brute force under a threshold.
The right strategy is a function of selectivity, and the engine or your code must decide per query.
Correlation makes it worse, or better
Selectivity is not the whole story. If the filter is anti-correlated with the query, winter coats filtered to swimwear, the matching nodes sit far from where the walk goes, and filter-aware traversal explores a huge region before it finds k matches. If the filter is correlated, invoices with type billing, the walk finds matches immediately. Denser graphs and per-partition indexes reduce that sensitivity.
Tenant isolation
| Design | Pros | Cons | Use when |
|---|---|---|---|
| Index per tenant | Exact isolation, simple deletes, small graphs | Many indexes, overhead, cold tenants | Few large tenants, strict compliance |
| Shared index plus a tenant filter | One index to operate | Filter problems for small tenants, noisy neighbors, leak risk if the filter is skipped | Many small tenants, lower compliance risk |
| Partition key | Small tenants grouped, large ones isolated | More routing logic | Most SaaS at scale: payload partitions, namespaces, multi-tenancy modes |
Tiny tenants should hit an exact search over their own ids. Large tenants get their own ANN structure. Missing the tenant filter is a data-leak incident, not a relevance bug.
Hybrid search
| Signal | Wins on | Fails on |
|---|---|---|
| BM25 | Exact tokens: error codes, SKUs, names, rare words | Synonyms, paraphrase, cross-language |
| Dense vectors | Paraphrase, intent, fuzzy meaning | Rare exact tokens, new jargon, numbers |
| Learned sparse | Expansion inside an inverted index | Model cost, index size |
Decisions
- 1
1. Query
- next2. Tenant, ACL, time filters
- 2
2. Tenant, ACL, time filters
- next3. Which legs?
- ?
3. Which legs?
- lexical4a. Filtered BM25 top 100
- dense4b. Filtered kNN top 100
- 4
4a. Filtered BM25 top 100
- next5. Fuse with RRF or weights
- 5
4b. Filtered kNN top 100
- next5. Fuse with RRF or weights
- 6
5. Fuse with RRF or weights
- next6. Cross-encoder re-rank
- 7
6. Cross-encoder re-rank
- next7. Return top 10
- 8
7. Return top 10
| Method | Formula shape | Pros | Cons |
|---|---|---|---|
| RRF | Sum of 1 / (k + rank), k often 60 | Scale-free, little tuning, robust | Ignores score margins. A weak second place counts like a strong one |
| Convex combination | a times normalized BM25 plus (1 minus a) times normalized cosine | Uses score strength, tunable | Needs normalization. Min-max is outlier sensitive. a needs labels |
| Raw sum | BM25 plus cosine | None | BM25, often 0 to 20, swamps cosine, 0 to 1 |
| Cascade | One leg, then the other re-scores | Cheap first stage | The first stage caps recall |
Elasticsearch can fuse with RRF or a linear combination. OpenSearch can normalize with min-max or L2 and combine with an arithmetic or harmonic mean. Weaviate, Qdrant, Vespa, and Postgres full text beside pgvector all have a hybrid pattern. Stale dense indexes, as a retrieval failure, are RAG Failure Modes — Hallucination, Stale Indexes, Evals & Grounding.
Sandbox: five strategies on one graph
The difference is when the filter is applied. post searches then drops. post+over asks for a longer list first. in-graph walks every node and admits only matches. strict walks only matches and strands. exact scores the matching ids and nothing else.
Press Run. Snippets must be self-contained — no network, files, or native modules.
Read the columns together. Recall without a full result list is a short page, not a good page. At 2 percent, exact is the plan you want, and post+over or in-graph have already walked essentially the whole corpus to get the same recall.
Sandbox: BM25, a toy dense leg, RRF
BM25 nails exact tokens and misses paraphrases. The toy embedding maps a few words onto shared axes so synonyms land together, and it has no axis for a rare code. RRF fuses by rank. A raw 0.5 and 0.5 blend lets BM25's wider range win. Min-max normalization lets the dense leg move the order.
Press Run. Snippets must be self-contained — no network, files, or native modules.
For E1234 the dense leg has no signal. Any fusion that keeps BM25 works, and the dense order is arbitrary. For the paraphrase, raw blending replays BM25 because those scores span about 0 to 4.5 while cosine spans 0 to 1. RRF and the min-max blend let the dense leg pull the disk-quota document up beside the paraphrase.
Interview Q&A
A filtered vector query returns 3 results when k is 10. Why?
Answer
Post-filtering. The engine fetched a top-k by vector similarity and then dropped non-matching hits. Pass the filter into the kNN clause, not a post-filter. Raise the candidate budget, or fall back to an exact search over the filtered ids when the filter is selective.
Why does a strict pre-filter break HNSW?
Answer
The graph was built over all nodes. If you only allow walking through matching nodes, the matching piece is often disconnected, so the walk strands near the entry point. Filter-aware traversal walks through non-matching nodes and only admits matches into the results.
How does Elasticsearch handle a filter inside a kNN query?
Answer
It applies the filter during the HNSW search so only matching documents are collected, and the result has k matches if they exist. If the filter is so restrictive that the graph would visit more nodes than the number of matching documents, Lucene switches to an exact search over the matches. A post-filter can still return fewer than k.
RRF versus a weighted sum?
Answer
RRF when you have no labeled data, the legs have incomparable scales, or you want a robust default. A weighted sum after normalization when you have labels to tune the weight and you want score strength, a very confident BM25 match, to count.
How would you design multi-tenant vector search for 10,000 tenants, a few of them huge?
Answer
Huge tenants get dedicated indexes or shards. Small tenants share an index partitioned by a tenant key, and their queries use an exact scan over their ids. Enforce the tenant filter on the server, and test for cross-tenant leakage.
When is hybrid search not worth it?
Answer
When queries are purely conversational paraphrase and the corpus has no identifiers, or when the latency budget cannot afford two legs. Measure on labeled queries. If BM25 adds no recall on your evaluation set, drop it.
What is ACORN?
Answer
A predicate-agnostic filtered HNSW approach. Build a denser graph and, during filtered search, expand two-hop neighborhoods through non-matching nodes so the matching nodes stay navigable. Weaviate ships a filter strategy in that family.
Why can two filters with the same selectivity behave differently?
Answer
Correlation. A filter whose matches sit next to the query is cheap for a filter-aware walk. A filter whose matches sit in a different region of the space forces a long exploration. Partitioning by the predicate removes that dependence.
Pitfalls
For 50, 10, and 2 percent, write the strategy you would ship and the one that returns a short list. Then write the fusion you would use for an error code and for a paraphrase, and say which score you refuse to add.