Why Hybrid Search?
Dense embeddings excel at capturing semantic meaning but may struggle with:- Exact keyword matches
- Proper nouns and domain-specific terminology
- Rare or out-of-vocabulary terms
- Precise keyword matching
- Token-level relevance
- Handling rare terms
How It Works
Qdrant allows you to store multiple named vectors per point. You can combine:- Dense vectors - traditional embeddings (e.g., from sentence transformers)
- Sparse vectors - token-based representations with weighted indices
Vector Configuration
Create a collection with both dense and sparse vectors:Inserting Data
Store both vector types for each point:Querying with Hybrid Search
Using Prefetch for Fusion
The recommended approach is to use the prefetch API with fusion:Fusion Strategies
RRF (Reciprocal Rank Fusion)
RRF (Reciprocal Rank Fusion)
The default and most commonly used fusion method. It combines rankings from multiple searches using reciprocal ranks:Where
k is a constant (typically 60) and rank_i is the position in each result list.Distribution-Based Score Fusion (DBSF)
Distribution-Based Score Fusion (DBSF)
Normalizes scores from different searches based on their statistical distribution before combining them:
Scoring Strategies
Separate Searches with Manual Combination
You can also perform separate searches and combine results in your application:Use Cases
E-commerce Search
Combine semantic understanding of product descriptions with exact SKU and brand name matching.
Legal Document Search
Find documents by meaning while ensuring specific legal terms and citations are matched.
Code Search
Search by semantic functionality while matching exact function names and identifiers.
Academic Papers
Semantic search for concepts combined with citation and author name matching.
Best Practices
Retrieval Limits
Retrieval Limits
Retrieve more candidates in prefetch (e.g., 20-100) than your final limit (e.g., 10) to ensure fusion has sufficient data to work with.
Vector Quality
Vector Quality
Ensure both dense and sparse vectors are generated from the same text to maintain consistency.
Testing Fusion Methods
Testing Fusion Methods
Test both RRF and DBSF with your specific dataset - performance varies by use case.
Monitoring
Monitoring
Track which vector type contributes more to final results to optimize your approach.
Performance Considerations
- Hybrid search requires two vector lookups, which increases query latency
- Use HNSW indexing for dense vectors to maintain fast search
- Sparse vector search uses inverted index, which is generally fast for high-dimensional sparse data
- Consider using filters in prefetch to reduce candidate sets before fusion
Related Topics
- Sparse Vectors - Deep dive into sparse vector representation
- Quantization - Reduce memory usage for large vector collections