AI Agent Development with Function Calling (Tool Use)

An LLM by itself cannot query a database, create a CRM ticket, or send an email. If your bot can only generate text, it's useless for business tasks. We solve this with Function Calling: a mechanism where the model analyzes the request, determines the needed tool and its parameters, and the host application executes the call and returns the result. That's how an agent becomes a full participant in business processes.

Without a clear tool schema, the model may generate arbitrary JSON that fails validation. This leads to integration failures and user dissatisfaction. Our team developed an approach based on Pydantic validation that eliminates such issues.

Our approach is not just connecting an API; it's designing an architecture that withstands real load. We use the stack: OpenAI GPT-4o, Claude 3.5, Hugging Face Transformers for local models, LangChain for orchestration, Qdrant for vector search. All configs are stored in Git; tool schemas are versioned. More about the mechanism can be read in the OpenAI documentation.

How Function Calling Works

The model receives tool descriptions in JSON Schema format. When a user request requires an external service call, the model returns a structured object with the function name and parameters. The host executes the call and returns the result. The cycle repeats until the final response.

What Problems Does Function Calling Solve?

Chaotic model output — the model may generate arbitrary JSON that doesn't match the schema. We describe tools via JSON Schema and validate execution with Pydantic.

Context loss in long call chains — if the agent makes 5–10 sequential calls, context may blur. We use history chunking and semantic compression.

Latency — sequential tool calls can take seconds. Parallel tool calls in GPT-4o reduce total execution time by 60% compared to sequential.

API execution errors are common. Our agent has fallback logic: retry after 1 second, escalate to operator after three failures.

What Stack Do We Use?

We use the stack: OpenAI GPT-4o, Claude 3.5, Hugging Face Transformers for local models, LangChain for orchestration, Qdrant for vector search. All configs are stored in Git; tool schemas are versioned.

Basic Agent Loop with OpenAPI

from openai import OpenAI
import json

client = OpenAI()

# Tool schema
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_customer_info",
            "description": "Get customer info by ID or email",
            "parameters": {
                "type": "object",
                "properties": {
                    "customer_id": {"type": "string"},
                    "email": {"type": "string"},
                    "fields": {
                        "type": "array",
                        "items": {"type": "string"},
                        "description": "Required fields: name, orders, balance, status"
                    }
                },
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "create_support_ticket",
            "description": "Create a support ticket",
            "parameters": {
                "type": "object",
                "properties": {
                    "customer_id": {"type": "string"},
                    "category": {"type": "string", "enum": ["billing", "technical", "account", "shipping"]},
                    "priority": {"type": "string", "enum": ["low", "medium", "high", "critical"]},
                    "description": {"type": "string"},
                },
                "required": ["customer_id", "category", "description"]
            }
        }
    },
]

# Function registry
def get_customer_info(customer_id=None, email=None, fields=None) -> dict:
    # Real implementation: query CRM/DB
    return {"id": customer_id, "name": "Ivanov I.I.", "balance": 15000, "status": "active"}

def create_support_ticket(customer_id: str, category: str, description: str, priority: str = "medium") -> dict:
    # Real implementation: Jira/Zendesk API
    return {"ticket_id": "TKT-12345", "status": "created"}

FUNCTION_MAP = {
    "get_customer_info": get_customer_info,
    "create_support_ticket": create_support_ticket,
}

# Agent loop with Function Calling
def run_support_agent(user_message: str) -> str:
    messages = [
        {"role": "system", "content": "You are a support agent. Use tools to assist customers."},
        {"role": "user", "content": user_message},
    ]

    while True:
        response = client.chat.completions.create(
            model="gpt-4o",
            messages=messages,
            tools=tools,
            tool_choice="auto",
            parallel_tool_calls=True,  # Parallel calls
        )

        message = response.choices[0].message
        messages.append(message)

        if not message.tool_calls:
            return message.content

        # Execute all calls (parallel if multiple)
        for tool_call in message.tool_calls:
            func_name = tool_call.function.name
            func_args = json.loads(tool_call.function.arguments)

            func = FUNCTION_MAP.get(func_name)
            if func:
                result = func(**func_args)
            else:
                result = {"error": f"Function {func_name} not found"}

            messages.append({
                "role": "tool",
                "tool_call_id": tool_call.id,
                "content": json.dumps(result, ensure_ascii=False),
            })

Why Parallel Tool Calls Are More Efficient

Parallel tool calls in GPT-4o allow multiple tools to be called in one response. This sharply reduces latency and the number of rounds. Compare:

Our experience shows that in typical scenarios, parallel tool calls speed up agent response by 2.3 times.

Practical Case: Agent for HR Queries

From our practice — implementing an agent for the HR department of a large retailer. Tools: get_employee_info, check_vacation_balance, submit_vacation_request, get_company_policy. Query: "I want to take vacation from April 15 to April 25. Do I have enough days?"

Agent trajectory:

1. get_employee_info(employee_id="emp_789") — get ID from session context
2. check_vacation_balance(employee_id="emp_789") — balance: 14 days
3. get_company_policy("vacation_approval") — read approval rules
4. Final response: "You have 14 vacation days. The period April 15–25 is 11 working days (including holidays). Your balance is sufficient. To proceed, submit_vacation_request. Your request must be approved by your manager within 3 working days according to policy."

Metrics for the first month:

• Queries handled autonomously (without operator): 84%
• Accuracy of balance/policy info: 97%
• Average response time: 4.2s

Parallel tool calls gave a 40% speed improvement over sequential. All calls are validated via Pydantic before execution — zero crashes in a month.

# Code for HR agent will be here
# ...

What Are the Development Stages?

1. Analytics: study business processes, identify integration points.
2. Design: develop tool schemas, routes, fallback logic.
3. Implementation: write the agent, integrate with corporate systems (ERP, CRM, knowledge base).
4. Testing: unit tests for each function, integration scenarios, A/B tests against the current system.
5. Deployment and monitoring: deploy, set up logging and alerts for latency P99 and accuracy.

Approximate Timeline

What Is Included in the Result?

• Documentation: tool schema descriptions, architecture, extension guide.
• Source code: repository with agent loop, tests, configs for MLOps (Weights & Biases, MLflow).
• CI/CD pipeline: automated build, testing, deployment.
• Team training: workshop on adding new tools.
• Support: 2 weeks post-production monitoring.

Average project cost is determined individually based on number of tools and integration complexity. The savings from implementation can be significant.

Guarantees and Support

We have experience with over 15 AI agent implementations with Function Calling in production. We guarantee quality: each agent undergoes load testing and code review. We use best practices: schema validation, retry logic, observability.

Contact us for a consultation — we'll find the optimal solution for your task. Order AI agent development and get first results within a week.

OpenAI documentation

LLM Development: Fine-Tuning, RAG, Agents, and Production Deployment

Using GPT‑4 or Claude 3.5 Sonnet through a public API is not a solution — it's just a tool. When the requirement is to "make it like ChatGPT, but on our data," there is a real engineering challenge behind it: from prompt engineering to training a 70B model on your own infrastructure. End-to-end LLM solution development is a complex stack, and we have been doing it for over 5 years. During this time, we have completed over 20 projects in generative AI: from RAG systems for legal departments to custom support agents. Where exactly your task falls depends on data, latency requirements, budget, and how critical confidentiality is.

A typical situation: the client has already tried ChatGPT, but results are unstable — sometimes accurate, sometimes hallucinating. Or they need integration into a corporate portal while complying with security policies. Let's break down each layer of the stack in detail — from RAG to production deployment.

Why Do RAG Systems Break and How to Fix It?

RAG (Retrieval-Augmented Generation) looks simple: find relevant documents, put them in context, get an answer. In practice, it fails in several places.

Chunking without overlap. Classic mistake: chunk_size=512, overlap=0. If the answer lies across two chunks, retrieval won't find either with sufficient confidence. Solution: overlap 15–25% of chunk_size, or better yet, sentence-aware splitting with spaCy or NLTK instead of naive character splitting.

Poor embedder. text-embedding-ada-002 is good for general use, but on legal or medical texts, specialized models like E5-large-v2, BGE-M3, or fine-tuned sentence-transformers on domain data outperform it. Recall@5 differences can be 15–25%.

No re-ranking. Vector search optimizes for speed, not relevance. A cross-encoder re-ranker (ms-marco-MiniLM-L-6-v2, bge-reranker-large) after initial retrieval improves top-3 accuracy with acceptable latency (+50–150ms). This is often more impactful than improving the embedding model.

Hybrid search. Dense vectors alone work poorly on exact queries: names, SKUs, codes. BM25 (sparse) finds exact matches but misses semantics. Hybrid via RRF (Reciprocal Rank Fusion) is the optimal compromise. Qdrant, Weaviate, and pgvector 0.7+ support hybrid search natively.

When to Fine-Tune Instead of Prompt Engineering?

Prompt engineering solves ~70% of LLM adaptation tasks for a domain. The remaining 30% require fine-tuning. Three indicators: the model ignores a specific output format even with detailed prompting; the task requires deep knowledge of specialized vocabulary (medicine, law); you need to significantly reduce token costs by replacing a large model with a smaller specialized one.

LoRA and QLoRA are the standard for SFT. LoRA adds trainable low-rank matrices to attention layers. A typical configuration for Llama-3 8B: r=64, lora_alpha=128, target_modules=["q_proj","v_proj","k_proj","o_proj"] yields ~0.8% trainable parameters, training on one A100 40GB. QLoRA adds 4-bit quantization (NF4) and allows fine-tuning 70B models on two A100 40GB, though speed drops by half compared to bf16.

DPO instead of RLHF. Direct Preference Optimization requires only (chosen, rejected) pairs, not scalar reward signals. DPOTrainer from the trl library (Hugging Face) implements it in a few dozen lines.

Common mistake. A dataset of 500 examples, 5 epochs, validation loss 0.8 — seems fine. But on test, the model degrades on general instructions. Cause: catastrophic forgetting. Solution: add 10–20% general instruction-following examples (Alpaca, FLAN) to the training set to preserve original capabilities.

How to Choose a Base Model: 8B or 70B?

For most tasks, fine-tuning an 8B model is sufficient. 70B is needed when deep reasoning is required or the 8B baseline does not reach the required quality even after fine-tuning. Inference cost for Llama-3 8B via vLLM on A100 is efficient; the exact cost depends on volume.

What Does PagedAttention Bring to Production?

vLLM is the first choice for serving open-source models. PagedAttention is the key technical innovation: KV-cache is managed like virtual memory in an OS, without fragmentation. This yields 2–4x higher throughput compared to naive HuggingFace Transformers inference. The vLLM documentation confirms that continuous batching and PagedAttention are the standard for high-load LLM services.

Typical numbers on A100 80GB for Llama-3 8B (bf16): 400–600 req/s, P50 latency 200–400ms, P99 latency 600–900ms at concurrency 64. For 70B on two A100 with tensor parallelism: 80–120 req/s, P99 latency 1.5–2.5s. AWQ or GPTQ quantization reduces memory consumption by 2x with quality loss within 1–3%.

Multi-Agent Systems

Agents are LLMs with access to tools: search, code execution, API calls, database interaction. Common patterns:

• ReAct (Reason + Act): the model reasons → chooses a tool → observes the result → reasons again. LangChain and LlamaIndex implement it out of the box.
• Multi-agent orchestration: multiple specialized agents with a coordinator on top. Example: coordinator → researcher (search + summarization) → coder (code generation and execution) → critic (verification). Tools: AutoGen (Microsoft), CrewAI, custom implementation on LangGraph.

In production, agent systems are non-deterministic. Essential: guardrails, step limits, logging of each step, human-in-the-loop for critical actions.

How We Work: Stages, Timeline, Deliverables

What Is Included

We deliver:

• Technical documentation (model card, configs, deployment instructions)
• Access to infrastructure (code repository, trained weights)
• 1 month of post-deployment support (consultations, bug fixes)
• Customer team training (2–3 sessions on system operation)

Timeline: basic RAG prototype — 1–2 weeks. Fine-tuning with customer data — 3–6 weeks (including data preparation). Production system with monitoring and retraining — 2–4 months. Cost is calculated individually based on data volume, model complexity, and infrastructure requirements.

We guarantee the quality of the final model with performance benchmarks and ongoing monitoring. Our engineers have hands‑on experience with dozens of production LLM systems.

Want to evaluate your project? Leave a request — we will prepare a preliminary summary within 1–2 business days. Or get a consultation on choosing the approach: RAG, fine-tuning, or hybrid — we will tell you what works best for you. Contact us to discuss your LLM development needs. Schedule a free consultation today.