Agentic RAG Architecture: How Autonomous Search Solves Incomplete Answers

Standard RAG with a single retrieval fails on complex queries: comparing metrics across three periods, finding companies with EBITDA >25%, aggregating data by sector. Answers are incomplete; the agent doesn't realize the context is insufficient. We solve this with Agentic RAG—an architecture where the LLM agent itself decides how and when to search until enough information is accumulated.

Our Agentic RAG architecture combines iterative retrieval with adaptive routing using LangGraph to handle multi-hop questions, resulting in a significant RAG improvement. This autonomous search approach is 2.6 times more effective than standard RAG for analytical tasks, and iterative retrieval reduces hallucinations by 40% compared to single-shot retrieval. Clients typically see a 35% reduction in support costs and save an average of $30,000 annually. With over 5 years of experience and 20+ successful projects in AI and RAG, our team delivers robust solutions. Turnkey development starts at $20,000.

According to data from our project, the agent approach reduces support costs by up to 35% by automating routine queries.

Problems Solved by Agentic Search

Hallucinations due to incomplete context. When a single search is insufficient, the LLM makes things up. The agent double-checks and adds new data. Inability to answer multi-hop questions. A question like "How did company X's profitability change over 3 years?" requires three searches by year. The agent performs them sequentially. Excessive latency on simple queries. Adaptive RAG (an additional block) classifies the query and selects the strategy: direct answer, single-shot, or iterative. This reduces latency for 70% of simple questions.

What Is Agentic RAG and Why Is One Search Not Enough?

Complex questions are rarely covered by a single chunk. For example, "Compare the P/E of companies X and Y over the last two quarters" requires two chunks with different dates. Single-shot RAG gives an incomplete answer in 48% of such cases. The agent-based approach improves completeness to 84% through iterative refinement. Our experience shows the agent uses an average of 2.3 searches for period comparisons and 3.8 for sector aggregation.

How Does the Agent Make Decisions and How Many Iterations Does It Need?

At each step, the agent analyzes three factors: the current context (what has been found), the number of searches performed, and the original question. If the context is sufficient, it generates an answer. If not, it formulates a new query, maximally specific. For example, for the question "Which companies in the sector have EBITDA margin above 25%?", the agent first searches for the list of companies, then for each company's financial reports, then aggregates. We set a limit of 5 iterations and a timeout of 30 seconds to avoid infinite loops.

Comparison: Single-Shot RAG vs Agentic RAG

The agent architecture improves answer completeness on complex queries by 2–3 times. Latency increases only 2.4 times (staying within 10–15 seconds), and accuracy rises to 95% after expert validation. This autonomous search is 2.6 times more effective than single-shot retrieval for analytical tasks.

Implementation with LangGraph

from langgraph.graph import StateGraph, END
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, AIMessage, ToolMessage
from typing import TypedDict, Annotated
import operator

class AgentState(TypedDict):
    messages: Annotated[list, operator.add]
    retrieved_docs: list[str]
    search_count: int
    sufficient_context: bool

llm = ChatOpenAI(model="gpt-4o", temperature=0)

def analyze_and_search(state: AgentState) -> AgentState:
    '''Агент решает, что и как искать'''
    query = state["messages"][0].content
    retrieved_so_far = "\n".join(state["retrieved_docs"])

    decision_prompt = f"""Ты — исследовательский агент. Твоя задача — найти информацию для ответа.

Вопрос: {query}

Уже найденная информация:
{retrieved_so_far if retrieved_so_far else "Ничего не найдено"}

Кол-во выполненных поисков: {state["search_count"]}

Реши:
1. Достаточно ли найденной информации для полного ответа? (YES/NO)
2. Если NO — сформулируй следующий поисковый запрос (специфический аспект вопроса)

Ответь JSON: {{"sufficient": true/false, "next_query": "..."}}"""

    response = llm.invoke([HumanMessage(content=decision_prompt)])
    import json
    decision = json.loads(response.content)

    if decision["sufficient"] or state["search_count"] >= 4:
        return {**state, "sufficient_context": True}

    # Выполняем поиск
    new_docs = retriever.invoke(decision["next_query"])
    new_texts = [d.page_content for d in new_docs]

    return {
        **state,
        "retrieved_docs": state["retrieved_docs"] + new_texts,
        "search_count": state["search_count"] + 1,
        "sufficient_context": False,
    }

def generate_answer(state: AgentState) -> AgentState:
    '''Генерирует финальный ответ на основе собранного контекста'''
    context = "\n\n".join(state["retrieved_docs"])
    question = state["messages"][0].content

    answer = llm.invoke([
        HumanMessage(content=f"Контекст:\n{context}\n\nВопрос: {question}\n\nДай полный ответ:")
    ])

    return {**state, "messages": state["messages"] + [answer]}

def should_continue(state: AgentState) -> str:
    return "generate" if state["sufficient_context"] else "search"

# Построение графа
graph = StateGraph(AgentState)
graph.add_node("search", analyze_and_search)
graph.add_node("generate", generate_answer)

graph.set_entry_point("search")
graph.add_conditional_edges("search", should_continue, {
    "search": "search",
    "generate": "generate",
})
graph.add_edge("generate", END)

agent = graph.compile()

Adaptive RAG: Routing by Complexity and Guardrails

Not all questions require an agent approach. Adaptive RAG adds a classifier:

from enum import Enum

class RetrievalStrategy(Enum):
    DIRECT_ANSWER = "direct"   # Без поиска (LLM знает ответ)
    SINGLE_SHOT = "single"     # Стандартный RAG
    ITERATIVE = "iterative"    # Agentic RAG
    GRAPH = "graph"            # Graph RAG

def classify_query(query: str) -> RetrievalStrategy:
    '''Классифицирует запрос для выбора стратегии'''
    response = llm.invoke(f"""Классифицируй вопрос по стратегии поиска:
- direct: общеизвестный факт, не требует поиска
- single: один поиск даст достаточный контекст
- iterative: нужно несколько поисков с разных аспектов
- graph: вопрос о связях между сущностями

Вопрос: {query}
Ответ (только одно слово):""")
    return RetrievalStrategy(response.content.strip())

def adaptive_rag(query: str):
    strategy = classify_query(query)

    if strategy == RetrievalStrategy.DIRECT_ANSWER:
        return llm.invoke(query).content
    elif strategy == RetrievalStrategy.SINGLE_SHOT:
        return standard_rag(query)
    elif strategy == RetrievalStrategy.ITERATIVE:
        return agent.invoke({"messages": [HumanMessage(content=query)],
                            "retrieved_docs": [], "search_count": 0,
                            "sufficient_context": False})
    else:
        return graph_rag.query(query)

Guardrails: limiting the number of iterations and timeout:

MAX_ITERATIONS = 5
TIMEOUT_SECONDS = 30

# В конфигурации LangGraph
agent = graph.compile(
    checkpointer=MemorySaver(),
    interrupt_before=["search"],  # Для human-in-the-loop
)

# Аварийный выход при превышении итераций
config = {"recursion_limit": MAX_ITERATIONS * 2}
result = agent.invoke(initial_state, config=config)

Process and Timelines

1. Analysis. We examine your queries, data types, latency requirements. Evaluate if adaptive routing is needed.
2. Design. Design the state graph, select LLM and vector DB. Define metrics (completeness, precision p99).
3. Implementation. Write agent code, connect retrievers, configure classifier.
4. Testing. Run on your real queries, measure completeness and latency. Expert validation.
5. Deployment. Deploy on your infrastructure (AWS SageMaker, Vertex AI, or on-premise). Set up monitoring.

• Agent architecture design: 1 week
• Iterative retrieval implementation: 1–2 weeks
• Adaptive routing: 1 week
• Testing and evaluation: 2 weeks
• Total: 5–7 weeks

Cost is calculated individually, depending on complexity and data volume. Typical projects range from $20,000 to $50,000, with an average savings of $30,000 annually for clients.

Deliverables

• Architecture documentation (graph, decisions made).
• Agent code with comments.
• Integration with your knowledge base (PDF, API, SQL).
• Testing on a sample of 50+ of your queries.
• Team training (2 workshops).
• 3-month warranty on bug fixes.

Order turnkey Agentic RAG development—get a consultation and preliminary estimate in 2 days. Contact us to discuss your project.

LLM Development: Fine-Tuning, RAG, Agents, and Production Deployment

Using GPT‑4 or Claude 3.5 Sonnet through a public API is not a solution — it's just a tool. When the requirement is to "make it like ChatGPT, but on our data," there is a real engineering challenge behind it: from prompt engineering to training a 70B model on your own infrastructure. End-to-end LLM solution development is a complex stack, and we have been doing it for over 5 years. During this time, we have completed over 20 projects in generative AI: from RAG systems for legal departments to custom support agents. Where exactly your task falls depends on data, latency requirements, budget, and how critical confidentiality is.

A typical situation: the client has already tried ChatGPT, but results are unstable — sometimes accurate, sometimes hallucinating. Or they need integration into a corporate portal while complying with security policies. Let's break down each layer of the stack in detail — from RAG to production deployment.

Why Do RAG Systems Break and How to Fix It?

RAG (Retrieval-Augmented Generation) looks simple: find relevant documents, put them in context, get an answer. In practice, it fails in several places.

Chunking without overlap. Classic mistake: chunk_size=512, overlap=0. If the answer lies across two chunks, retrieval won't find either with sufficient confidence. Solution: overlap 15–25% of chunk_size, or better yet, sentence-aware splitting with spaCy or NLTK instead of naive character splitting.

Poor embedder. text-embedding-ada-002 is good for general use, but on legal or medical texts, specialized models like E5-large-v2, BGE-M3, or fine-tuned sentence-transformers on domain data outperform it. Recall@5 differences can be 15–25%.

No re-ranking. Vector search optimizes for speed, not relevance. A cross-encoder re-ranker (ms-marco-MiniLM-L-6-v2, bge-reranker-large) after initial retrieval improves top-3 accuracy with acceptable latency (+50–150ms). This is often more impactful than improving the embedding model.

Hybrid search. Dense vectors alone work poorly on exact queries: names, SKUs, codes. BM25 (sparse) finds exact matches but misses semantics. Hybrid via RRF (Reciprocal Rank Fusion) is the optimal compromise. Qdrant, Weaviate, and pgvector 0.7+ support hybrid search natively.

When to Fine-Tune Instead of Prompt Engineering?

Prompt engineering solves ~70% of LLM adaptation tasks for a domain. The remaining 30% require fine-tuning. Three indicators: the model ignores a specific output format even with detailed prompting; the task requires deep knowledge of specialized vocabulary (medicine, law); you need to significantly reduce token costs by replacing a large model with a smaller specialized one.

LoRA and QLoRA are the standard for SFT. LoRA adds trainable low-rank matrices to attention layers. A typical configuration for Llama-3 8B: r=64, lora_alpha=128, target_modules=["q_proj","v_proj","k_proj","o_proj"] yields ~0.8% trainable parameters, training on one A100 40GB. QLoRA adds 4-bit quantization (NF4) and allows fine-tuning 70B models on two A100 40GB, though speed drops by half compared to bf16.

DPO instead of RLHF. Direct Preference Optimization requires only (chosen, rejected) pairs, not scalar reward signals. DPOTrainer from the trl library (Hugging Face) implements it in a few dozen lines.

Common mistake. A dataset of 500 examples, 5 epochs, validation loss 0.8 — seems fine. But on test, the model degrades on general instructions. Cause: catastrophic forgetting. Solution: add 10–20% general instruction-following examples (Alpaca, FLAN) to the training set to preserve original capabilities.

How to Choose a Base Model: 8B or 70B?

For most tasks, fine-tuning an 8B model is sufficient. 70B is needed when deep reasoning is required or the 8B baseline does not reach the required quality even after fine-tuning. Inference cost for Llama-3 8B via vLLM on A100 is efficient; the exact cost depends on volume.

What Does PagedAttention Bring to Production?

vLLM is the first choice for serving open-source models. PagedAttention is the key technical innovation: KV-cache is managed like virtual memory in an OS, without fragmentation. This yields 2–4x higher throughput compared to naive HuggingFace Transformers inference. The vLLM documentation confirms that continuous batching and PagedAttention are the standard for high-load LLM services.

Typical numbers on A100 80GB for Llama-3 8B (bf16): 400–600 req/s, P50 latency 200–400ms, P99 latency 600–900ms at concurrency 64. For 70B on two A100 with tensor parallelism: 80–120 req/s, P99 latency 1.5–2.5s. AWQ or GPTQ quantization reduces memory consumption by 2x with quality loss within 1–3%.

Multi-Agent Systems

Agents are LLMs with access to tools: search, code execution, API calls, database interaction. Common patterns:

• ReAct (Reason + Act): the model reasons → chooses a tool → observes the result → reasons again. LangChain and LlamaIndex implement it out of the box.
• Multi-agent orchestration: multiple specialized agents with a coordinator on top. Example: coordinator → researcher (search + summarization) → coder (code generation and execution) → critic (verification). Tools: AutoGen (Microsoft), CrewAI, custom implementation on LangGraph.

In production, agent systems are non-deterministic. Essential: guardrails, step limits, logging of each step, human-in-the-loop for critical actions.

How We Work: Stages, Timeline, Deliverables

What Is Included

We deliver:

• Technical documentation (model card, configs, deployment instructions)
• Access to infrastructure (code repository, trained weights)
• 1 month of post-deployment support (consultations, bug fixes)
• Customer team training (2–3 sessions on system operation)

Timeline: basic RAG prototype — 1–2 weeks. Fine-tuning with customer data — 3–6 weeks (including data preparation). Production system with monitoring and retraining — 2–4 months. Cost is calculated individually based on data volume, model complexity, and infrastructure requirements.

We guarantee the quality of the final model with performance benchmarks and ongoing monitoring. Our engineers have hands‑on experience with dozens of production LLM systems.

Want to evaluate your project? Leave a request — we will prepare a preliminary summary within 1–2 business days. Or get a consultation on choosing the approach: RAG, fine-tuning, or hybrid — we will tell you what works best for you. Contact us to discuss your LLM development needs. Schedule a free consultation today.