Problem: Classifying Intent in the First Seconds of a Call
A customer calls tech support and says "I got an internet bill." Traditional DTMF menus force them to press buttons, irritating them and wasting time. Call center operator load increases, and average handling time rises. We solve this with an AI intent classifier that determines the call's purpose from natural speech in seconds and routes to the right service. Routing accuracy >90% is the key performance metric for AI-IVR. The system analyzes not just individual keywords but the full context of the phrase, distinguishing "pay the bill" from "check the amount." In our practice, deploying such a classifier reduced first-line support load by 35%. According to Gartner, IVR automation cuts call center costs by up to 40%.
How AI Classification Improves Routing Accuracy
Classic IVR uses DTMF menus and simple keyword triggers. An AI approach based on LLMs and embeddings understands intent even from a single phrase with context. We apply a multi-level taxonomy: first determine the main category (billing, technical, contract), then the subcategory, and extract entities (account number, address).
from pydantic import BaseModel
class IntentClassification(BaseModel):
primary_intent: str # main intent
secondary_intent: str = None # refinement
entities: dict = {} # extracted entities
confidence: float
requires_clarification: bool = False
# Intent taxonomy (example for telecom)
INTENT_TAXONOMY = {
"billing": {
"subcategories": ["invoice", "payment", "debt", "tariff_change"],
"examples": ["how much do I owe", "pay the bill", "change plan"]
},
"technical": {
"subcategories": ["no_internet", "slow_speed", "tv_issue", "router"],
"examples": ["internet not working", "slow speed", "television"]
},
"contract": {
"subcategories": ["new_connection", "cancellation", "address_change"],
"examples": ["connect", "cancel contract", "move"]
}
}
async def classify_caller_intent(
utterance: str,
taxonomy: dict
) -> IntentClassification:
taxonomy_description = "\n".join(
f"{cat}: {', '.join(data['examples'][:3])}"
for cat, data in taxonomy.items()
)
response = await client.chat.completions.create(
model="gpt-4o-mini",
messages=[{
"role": "system",
"content": f"""Classify the caller's intent.
Categories and examples:
{taxonomy_description}
Return JSON: {{
"primary_intent": "...",
"secondary_intent": "...",
"entities": {{}},
"confidence": 0.0-1.0,
"requires_clarification": false
}}"""
}, {"role": "user", "content": utterance}],
response_format={"type": "json_object"}
)
data = json.loads(response.choices[0].message.content)
return IntentClassification(**data)
Why Multi-Level Taxonomy Beats Flat Classification
Flat classification (e.g., 20 categories on one level) yields about 75-80% accuracy due to class overlap. Hierarchical taxonomy with main and subordinate intents boosts accuracy to 90-95%. It also asks clarifying questions only in ambiguous cases, without overloading the customer with extra dialogues. For context enrichment, we use RAG, especially useful for rare queries. Additionally, we fine-tune the base LLM on your data, adapting the model to your business specifics.
Handling Ambiguous Intents
CLARIFICATION_TEMPLATES = {
"billing_vs_technical": "Could you clarify – are you calling about billing or a technical issue?",
"new_vs_existing": "Are you already our customer or do you want to sign up?",
"internet_vs_tv": "What exactly isn't working – internet or television?",
}
async def handle_ambiguous_intent(
call: IncomingCall,
classification: IntentClassification
) -> IntentClassification:
if not classification.requires_clarification:
return classification
# Determine the appropriate clarifying question
clarification = determine_clarification_question(
classification.primary_intent
)
await call.say(clarification)
response = await call.listen(timeout_sec=8)
return await classify_caller_intent(response, INTENT_TAXONOMY)
Testing and Monitoring Accuracy
async def evaluate_ivr_accuracy(test_set: list[dict]) -> dict:
"""Test the classifier on a test set"""
correct = 0
total = len(test_set)
for test_case in test_set:
result = await classify_caller_intent(
test_case["utterance"], INTENT_TAXONOMY
)
if result.primary_intent == test_case["expected_intent"]:
correct += 1
accuracy = correct / total
return {
"accuracy": accuracy,
"correct": correct,
"total": total,
"target_met": accuracy >= 0.90 # 90% — target metric
}
Comparison: Flat vs. Hierarchical Classification
| Parameter | Flat Classification | Hierarchical + Clarification |
|---|---|---|
| Accuracy | 75-80% | 90-95% |
| Processing Time | <0.5 sec | <1 sec |
| Need for Call Center | ~30% of calls | <10% (only complex cases) |
| Adaptation to New Categories | Retrain whole model | Add subcategory without retrain |
Technology Stack and Implementation Timeline
| Component | Technology | Development Duration |
|---|---|---|
| Intent Classifier | GPT-4o, LLaMA 3 (fine-tuned) | 1-2 weeks |
| Vectorization & Search | OpenAI embeddings (1536-dim), pgvector | 3-5 days |
| Clarification | LangChain, YAML templates | 1 week |
| API Service | FastAPI, Triton Inference Server | 1-2 weeks |
| IVR Integration | REST API / WebSocket | 1-2 weeks |
Common Mistakes in AI-IVR Implementation
- Taxonomy too broad – more than 15 main categories reduce accuracy. Optimum is 5-7.
- Ignoring entities – without extracting order number or tariff, routing remains imprecise.
- No A/B testing – launching without comparative analysis against current IVR leads to unexpected drops.
- Underestimating latency – if classification takes >2 seconds, customers hang up.
AI-IVR Implementation Process
- Analysis of current IVR logs – collect typical queries, identify categories (1-2 days).
- Taxonomy design – jointly with your experts (1-2 days).
- Classifier development – configure LLM, embeddings, clarification (1-2 weeks).
- Integration with IVR platform – via REST API or WebSocket (1-2 weeks).
- Load testing – verify p99 latency and accuracy on production data (3-5 days).
- Pilot launch – parallel operation with monitoring (1-2 weeks).
What's Included
- Classification model – trained on your data, with taxonomy documentation.
- API service – wrapper for calling from IVR, including error handling.
- Test set – annotated calls for accuracy verification.
- Support instructions – adding new categories, updating the model.
- Operator training – how to handle transfers from the AI classifier.
Timeline and Pricing
Basic classifier and testing – 2-3 weeks. Full integration with IVR platform, model training on your data, and scenario setup – 4-6 weeks. Pricing is tailored to your project – we'll estimate after understanding your specifics.
We guarantee accuracy >90% at the testing stage. Reduce call center costs by up to 40% by lowering operator load. Savings on call routing budget reach 30%. Our experience – over 5 years in AI solutions for voice interfaces.
Contact us to discuss your AI-IVR architecture. We'll audit your current system and propose an implementation plan. Request a pilot project – test the classifier on your logs in 2 weeks.