Hybrid search is the standard way to do retrieval now. BM25 finds exact words. Vector KNN finds similar meaning. You run both, then merge the two rankings with reciprocal rank fusion (RRF). It is in every tutorial, and it works.
But it can break in a way nothing warns you about. One of the two legs can return nothing, and the merged result still looks fine.
I hit this in a search system I built over an internal issue tracker. About 100,000 tickets. Engineers use it to find the ticket where someone already fixed the same problem. Here is what went wrong, and how I fixed it.
The setup
Everything runs on one RTX A1000 with 4 GB of VRAM. That limit shaped the whole design:
- Gemma-4 E2B-it, 4-bit nf4 quantised with bitsandbytes. It structures tickets and reads image attachments.
- multilingual-e5-large for embeddings, 1024 dimensions
- Elasticsearch 9.x for both the keyword index and the vectors
When a ticket is added, the model splits it into fields: problem, root cause, solution, keywords. When someone searches, BM25 and vector KNN both run, and RRF merges the two rankings.
The trap: long queries
One search path does not use a short phrase. You upload a ticket, the model summarises it, and that whole summary becomes the query. These summaries are long. The median is 866 tokens.
Now look at this BM25 setting. It looks completely normal:
"minimum_should_match": "30%"
For “login fails after update” it is right. Six words, two must match.
For an 866-token query it matches nothing. "30%" of a 300-word query means a
document must share 90 words before Elasticsearch will even look at it.
No document shares 90 words. So BM25 returns an empty list. RRF then merges one full ranking with one empty one. What comes out is plain vector search. It is still called hybrid.
Why this is worth knowing
The threshold is not the interesting part. What matters is how this looks from outside.
The results page is full. The scores look normal. The results are even useful, because vector search on its own is not bad search. But it stops finding the things BM25 is there for: an exact error string, a version number, a component name. The embedding blurs those into similar-looking words.
A hybrid system with one dead leg looks the same as a healthy one. No error. No empty page. No change in speed. All the usual alerts stay green.
If a component returns junk, you notice. If it returns nothing, you do not. Its weight just moves to whatever else is in the merge.
The fix
Make the threshold depend on how long the query is:
def _min_should_match(query_text: str) -> str:
term_count = len(query_text.split())
if term_count <= 15:
return "30%" # short query — the normal default is fine
if term_count <= 60:
return "2<-25%" # 2 words or fewer: match all. Longer: match 25%
return "4" # very long — use a fixed count, not a ratio
The last line matters most. If you keep using a percentage, longer queries need more and more matching words. A 2,000-token query at 25% still needs hundreds of them. That is the same bug again with bigger numbers. At some length you have to stop using a ratio. Four good word matches in a very long query is already a strong signal.
Elasticsearch supports the "2<-25%" form directly. It means: with 2 words or fewer,
match all of them; above that, match 25%. It is in the docs, and almost nobody uses
it.
How to check your own
Add one line before you merge:
logger.debug("bm25=%d knn=%d", len(bm25_hits), len(knn_hits))
Every leg should say how many results it returned. Zero is the case you care about, and it is the one you cannot see in the output.
If you run hybrid search and have never checked these counts on your longest queries, go and look. It takes a minute.