Hybrid Search Implementation (Vector + Full-Text) for RAG

When implementing RAG systems, a common dilemma is how to find a document both by meaning and by exact number. Hybrid search—a combination of vector (dense) and full-text (sparse/BM25) retrieval followed by result fusion—solves this problem. In practice, hybrid search consistently outperforms either method alone on most corporate datasets. For example, on one project hybrid search (RRF) improved MRR@5 by 12% relative to pure dense search while maintaining high recall for exact terms. We implement such solutions turnkey, with guaranteed retrieval quality on your data. Project cost typically ranges from $5,000 to $10,000 depending on dataset size and complexity. Request a consultation on hybrid search implementation and get a project assessment.

Why Dense Search Alone Is Not Enough

Dense embeddings average semantics—that's both a strength and a weakness. A query like "contract No. DA-2023-451" will have high cosine similarity with contracts in general, but not with the specific document by number. BM25 finds the exact string "DA-2023-451" instantly.

• Dense search performs poorly for: exact numbers (contract, SKU, serial number), abbreviations and specific acronyms, rare technical terms, queries for exact quote search.
• BM25 performs poorly for: paraphrased queries (synonyms), semantically similar concepts with different words, cross-lingual queries, vague descriptions ("something about payment after delivery").

Why Hybrid Search Is Better Than Dense or BM25 Alone

Combining the two approaches yields synergy: dense covers semantics, BM25 covers exact matches. The real-world case below shows that hybrid RRF (without reranker) outperforms dense+reranker in MRR@5 (0.83 vs 0.80) and NDCG@5 (0.81 vs 0.77). Meanwhile, hybrid+reranker achieves 0.89/0.87. In other words, hybrid search implementation achieves a balance between semantic similarity and exact keyword matching. According to our A/B test on 400 queries, hybrid RRF outperforms dense+reranker by a factor of 1.04 in MRR@5. For many tasks, this eliminates the need for an expensive reranker.

Fusion Algorithms

Reciprocal Rank Fusion (RRF)—the most robust method. RRF is a fusion method proposed by Cormack et al. (2009)—see more on Wikipedia.

from collections import defaultdict

def reciprocal_rank_fusion(
    dense_results: list[tuple],   # [(doc_id, score), ...]
    sparse_results: list[tuple],
    k: int = 60  # RRF constant (typically 60)
) -> list[tuple]:
    """
    RRF score = sum(1 / (k + rank_i)) across all lists
    k=60 standard value (Cormack et al.)
    """
    scores = defaultdict(float)

    for rank, (doc_id, _) in enumerate(dense_results, 1):
        scores[doc_id] += 1 / (k + rank)

    for rank, (doc_id, _) in enumerate(sparse_results, 1):
        scores[doc_id] += 1 / (k + rank)

    return sorted(scores.items(), key=lambda x: -x[1])

Relative Score Fusion (RSF)—normalized combination:

def relative_score_fusion(
    dense_results: list[tuple],
    sparse_results: list[tuple],
    alpha: float = 0.5  # Weight for dense
) -> list[tuple]:
    """Normalizes scores to [0,1] and weights them"""
    scores = defaultdict(float)

    # Normalize dense
    if dense_results:
        max_d = max(s for _, s in dense_results)
        min_d = min(s for _, s in dense_results)
        for doc_id, score in dense_results:
            norm = (score - min_d) / (max_d - min_d + 1e-8)
            scores[doc_id] += alpha * norm

    # Normalize sparse
    if sparse_results:
        max_s = max(s for _, s in sparse_results)
        min_s = min(s for _, s in sparse_results)
        for doc_id, score in sparse_results:
            norm = (score - min_s) / (max_s - min_s + 1e-8)
            scores[doc_id] += (1 - alpha) * norm

    return sorted(scores.items(), key=lambda x: -x[1])

Fusion Algorithm Comparison

SPLADE: Advanced Sparse Encoder

SPLADE (Sparse Lexical and Expansion Model) generates sparse vectors with lexical expansion—the model learns to "expand" the query with synonyms and related terms. According to the BEIR benchmark, SPLADE outperforms BM25 by 1.2–1.5 times in NDCG@10.

from fastembed import SparseTextEmbedding

sparse_model = SparseTextEmbedding(
    model_name="prithivida/Splade_PP_en_v1"
)

def encode_sparse(text: str) -> dict:
    """Returns sparse vector {token_id: weight}"""
    output = list(sparse_model.embed([text]))[0]
    return {
        "indices": output.indices.tolist(),
        "values": output.values.tolist(),
    }

SPLADE outperforms BM25 on most BEIR benchmarks. For Russian, we recommend the model naver/efficient-splade-VI-BT-large-query or multilingual variants.

Implementation with Qdrant (Practical Example)

from qdrant_client import QdrantClient
from qdrant_client.models import (
    SparseVector, Prefetch, FusionQuery, Fusion,
    NamedVector, NamedSparseVector
)
from fastembed import TextEmbedding, SparseTextEmbedding

dense_model = TextEmbedding("BAAI/bge-m3")  # Multilingual dense
sparse_model = SparseTextEmbedding("prithivida/Splade_PP_en_v1")
client = QdrantClient(url="http://localhost:6333")

def hybrid_search(query: str, top_k: int = 5) -> list[dict]:
    # Dense embedding
    dense_vec = list(dense_model.embed([query]))[0].tolist()

    # Sparse embedding
    sparse_output = list(sparse_model.embed([query]))[0]
    sparse_vec = SparseVector(
        indices=sparse_output.indices.tolist(),
        values=sparse_output.values.tolist()
    )

    results = client.query_points(
        collection_name="hybrid_docs",
        prefetch=[
            Prefetch(query=dense_vec, using="dense", limit=50),
            Prefetch(query=sparse_vec, using="sparse", limit=50),
        ],
        query=FusionQuery(fusion=Fusion.RRF),
        limit=top_k,
        with_payload=True,
    )

    return [
        {"text": r.payload["text"], "source": r.payload["source"], "score": r.score}
        for r in results.points
    ]

Practical Case: Alpha Impact on Retrieval Quality

From our practice: on a project with 12,000 corporate knowledge base documents (contracts, regulations, FAQs), we tested 400 queries of various types. Results:

Hybrid RRF without reranker already beats dense+reranker. The combination hybrid+reranker yields the best result. For comparison, using SPLADE as a sparse encoder gives an MRR@5 improvement of about 0.03–0.05 over BM25 with the same fusion method.

How to Tune RRF Fusion on Your Dataset?

Optimal k for RRF: k=60 is an empirically robust value. Too small k (10–20) gives large weight to top positions. Too large (100+) levels out differences between positions. On real data, test k∈{20, 40, 60, 80} on a validation set. For RSF, tune alpha from 0.3 to 0.7 in steps of 0.1.

Step-by-Step Hybrid Search Implementation Process

1. Audit current retrieval scheme: analyze used embeddings, vector DB stack, and quality metrics.
2. Select and configure sparse encoder: install SPLADE or other sparse encoder suited to your language and domain.
3. Integrate dual search: set up indexing of dense and sparse vectors in Qdrant/Pinecone/Weaviate.
4. Implement fusion: deploy RRF or RSF with initial parameters (k=60, alpha=0.5).
5. Test and optimize: run your queries, tune parameters using MRR/NDCG metrics.
6. Document and hand over: describe the process, train the team, deliver code and configs.

What's Included in the Project

• Integration code for hybrid search into your RAG system.
• Configuration files for Qdrant/Pinecone.
• Documentation on setup and operation.
• Team training (2-hour webinar).
• Retrieval quality guarantee (metrics measured before/after).
• One month post-project support.

Contact us for a free project assessment. Get a consultation on hybrid search implementation and improve your RAG system's retrieval quality.

LLM Development: Fine-Tuning, RAG, Agents, and Production Deployment

Using GPT‑4 or Claude 3.5 Sonnet through a public API is not a solution — it's just a tool. When the requirement is to "make it like ChatGPT, but on our data," there is a real engineering challenge behind it: from prompt engineering to training a 70B model on your own infrastructure. End-to-end LLM solution development is a complex stack, and we have been doing it for over 5 years. During this time, we have completed over 20 projects in generative AI: from RAG systems for legal departments to custom support agents. Where exactly your task falls depends on data, latency requirements, budget, and how critical confidentiality is.

A typical situation: the client has already tried ChatGPT, but results are unstable — sometimes accurate, sometimes hallucinating. Or they need integration into a corporate portal while complying with security policies. Let's break down each layer of the stack in detail — from RAG to production deployment.

Why Do RAG Systems Break and How to Fix It?

RAG (Retrieval-Augmented Generation) looks simple: find relevant documents, put them in context, get an answer. In practice, it fails in several places.

Chunking without overlap. Classic mistake: chunk_size=512, overlap=0. If the answer lies across two chunks, retrieval won't find either with sufficient confidence. Solution: overlap 15–25% of chunk_size, or better yet, sentence-aware splitting with spaCy or NLTK instead of naive character splitting.

Poor embedder. text-embedding-ada-002 is good for general use, but on legal or medical texts, specialized models like E5-large-v2, BGE-M3, or fine-tuned sentence-transformers on domain data outperform it. Recall@5 differences can be 15–25%.

No re-ranking. Vector search optimizes for speed, not relevance. A cross-encoder re-ranker (ms-marco-MiniLM-L-6-v2, bge-reranker-large) after initial retrieval improves top-3 accuracy with acceptable latency (+50–150ms). This is often more impactful than improving the embedding model.

Hybrid search. Dense vectors alone work poorly on exact queries: names, SKUs, codes. BM25 (sparse) finds exact matches but misses semantics. Hybrid via RRF (Reciprocal Rank Fusion) is the optimal compromise. Qdrant, Weaviate, and pgvector 0.7+ support hybrid search natively.

When to Fine-Tune Instead of Prompt Engineering?

Prompt engineering solves ~70% of LLM adaptation tasks for a domain. The remaining 30% require fine-tuning. Three indicators: the model ignores a specific output format even with detailed prompting; the task requires deep knowledge of specialized vocabulary (medicine, law); you need to significantly reduce token costs by replacing a large model with a smaller specialized one.

LoRA and QLoRA are the standard for SFT. LoRA adds trainable low-rank matrices to attention layers. A typical configuration for Llama-3 8B: r=64, lora_alpha=128, target_modules=["q_proj","v_proj","k_proj","o_proj"] yields ~0.8% trainable parameters, training on one A100 40GB. QLoRA adds 4-bit quantization (NF4) and allows fine-tuning 70B models on two A100 40GB, though speed drops by half compared to bf16.

DPO instead of RLHF. Direct Preference Optimization requires only (chosen, rejected) pairs, not scalar reward signals. DPOTrainer from the trl library (Hugging Face) implements it in a few dozen lines.

Common mistake. A dataset of 500 examples, 5 epochs, validation loss 0.8 — seems fine. But on test, the model degrades on general instructions. Cause: catastrophic forgetting. Solution: add 10–20% general instruction-following examples (Alpaca, FLAN) to the training set to preserve original capabilities.

How to Choose a Base Model: 8B or 70B?

For most tasks, fine-tuning an 8B model is sufficient. 70B is needed when deep reasoning is required or the 8B baseline does not reach the required quality even after fine-tuning. Inference cost for Llama-3 8B via vLLM on A100 is efficient; the exact cost depends on volume.

What Does PagedAttention Bring to Production?

vLLM is the first choice for serving open-source models. PagedAttention is the key technical innovation: KV-cache is managed like virtual memory in an OS, without fragmentation. This yields 2–4x higher throughput compared to naive HuggingFace Transformers inference. The vLLM documentation confirms that continuous batching and PagedAttention are the standard for high-load LLM services.

Typical numbers on A100 80GB for Llama-3 8B (bf16): 400–600 req/s, P50 latency 200–400ms, P99 latency 600–900ms at concurrency 64. For 70B on two A100 with tensor parallelism: 80–120 req/s, P99 latency 1.5–2.5s. AWQ or GPTQ quantization reduces memory consumption by 2x with quality loss within 1–3%.

Multi-Agent Systems

Agents are LLMs with access to tools: search, code execution, API calls, database interaction. Common patterns:

• ReAct (Reason + Act): the model reasons → chooses a tool → observes the result → reasons again. LangChain and LlamaIndex implement it out of the box.
• Multi-agent orchestration: multiple specialized agents with a coordinator on top. Example: coordinator → researcher (search + summarization) → coder (code generation and execution) → critic (verification). Tools: AutoGen (Microsoft), CrewAI, custom implementation on LangGraph.

In production, agent systems are non-deterministic. Essential: guardrails, step limits, logging of each step, human-in-the-loop for critical actions.

How We Work: Stages, Timeline, Deliverables

What Is Included

We deliver:

• Technical documentation (model card, configs, deployment instructions)
• Access to infrastructure (code repository, trained weights)
• 1 month of post-deployment support (consultations, bug fixes)
• Customer team training (2–3 sessions on system operation)

Timeline: basic RAG prototype — 1–2 weeks. Fine-tuning with customer data — 3–6 weeks (including data preparation). Production system with monitoring and retraining — 2–4 months. Cost is calculated individually based on data volume, model complexity, and infrastructure requirements.

We guarantee the quality of the final model with performance benchmarks and ongoing monitoring. Our engineers have hands‑on experience with dozens of production LLM systems.

Want to evaluate your project? Leave a request — we will prepare a preliminary summary within 1–2 business days. Or get a consultation on choosing the approach: RAG, fine-tuning, or hybrid — we will tell you what works best for you. Contact us to discuss your LLM development needs. Schedule a free consultation today.