AI Auto-Dialer for Debt Collection: Voice Bot for Debt Recovery

AI Auto-Dialer for Debt Collection: Voice Bot for Debt Recovery

Collection departments spend up to 70% of their time on dialing and compliance checks under 230-FZ. AI auto-dialer replaces operators in early overdue stages (DPD 1–30): the voice bot informs about the debt, offers payment options, and records Promise to Pay (PTP). A typical scenario: an operator spends a minute on greetings and data verification, and calls after 18:00 are prohibited. An AI bot processes 1000 contacts in 2–4 hours without breaks or errors. Moreover, the voice bot adapts to the debtor's accent and intonation, increasing PTP rates by 15–20% according to our data. Our experience: over 5 years in AI solutions for fintech, 30+ deployed systems. In one project for an MFO with a portfolio of 30,000 active debtors, deploying AI calling reduced call center load by 60% in the first month, and PTP rate increased from 12% to 18%.

How AI Auto-Dialer Complies with 230-FZ?

Law 230-FZ strictly regulates the frequency and timing of calls to debtors. The system automatically checks each connection:

from datetime import time

CALL_RESTRICTIONS = {
    'weekdays': (time(8, 0), time(22, 0)),
    'weekends': (time(9, 0), time(20, 0)),
    'max_calls_per_day': 1,
    'max_calls_per_week': 2,
    'max_calls_per_month': 8,
}

async def check_call_allowed(debtor_id: str) -> tuple[bool, str]:
    now = datetime.now()
    contact_history = await db.get_contact_history(debtor_id)
    current_time = now.time()
    window = CALL_RESTRICTIONS['weekdays'] if now.weekday() < 5 else CALL_RESTRICTIONS['weekends']
    if not (window[0] <= current_time <= window[1]):
        return False, 'outside_allowed_hours'
    today_calls = sum(1 for c in contact_history if c['date'].date() == now.date())
    if today_calls >= CALL_RESTRICTIONS['max_calls_per_day']:
        return False, 'daily_limit_exceeded'
    if await is_in_stoplist(debtor_id):
        return False, 'in_stoplist'
    return True, 'allowed'

Additionally, the compliance module tracks changes in the law and updates rules without stopping the service. We guarantee that the system always meets the current requirements of 230-FZ.

For speech recognition we use Whisper, and the dialog engine is built on GPT-4o-mini with fine-tuning on historical recordings. p99 latency does not exceed 800 ms thanks to optimized pipeline and batch processing.

Why AI is More Efficient Than Manual Calls?

AI handles repetitive scenarios more consistently, does not get tired, and avoids human errors like missing a stop-list. Additionally, the system logs every interaction for subsequent audit.

How Scripts Adapt to DPD Stage?

COLLECTION_SCRIPTS = {
    'dpd_1_7': """
        Hello, {name}! This is {creditor}.
        Your contract {contract_id} has an overdue payment of {amount} rubles.
        Please pay by {payment_deadline}.
        Do you plan to pay soon?
    """,
    'dpd_8_30': """
        Hello, {name}. This is a call from {creditor}.
        Your debt under contract {contract_id} is {amount} rubles,
        overdue {dpd} days. Penalties are accruing at {penalty_rate}% per day.
        We are ready to consider a convenient repayment plan for you.
        Can you pay today?
    """,
}

Adapting scripts to a specific portfolio (amounts, tone, legal wording) takes 1–2 days. We use few-shot prompts for quick setup without model retraining.

Capturing PTP and Installment Plans

@dataclass
class PromiseToPay:
    debtor_id: str
    contract_id: str
    promised_amount: float
    promised_date: date
    call_id: str
    confirmed: bool = False

async def process_payment_commitment(
    session: dict,
    user_response: str
) -> PromiseToPay | None:
    extraction = await client.chat.completions.create(
        model='gpt-4o-mini',
        messages=[{
            'role': 'system',
            'content': 'Extract payment promise from text. JSON: {"date": "DD.MM", "amount": N, "full": bool}'
        }, {'role': 'user', 'content': user_response}],
        response_format={'type': 'json_object'}
    )
    data = json.loads(extraction.choices[0].message.content)
    if data.get('date'):
        ptp = PromiseToPay(
            debtor_id=session['debtor_id'],
            contract_id=session['contract_id'],
            promised_amount=data.get('amount', session['total_debt']),
            promised_date=parse_date(data['date']),
            call_id=session['call_id']
        )
        await db.save_ptp(ptp)
        return ptp
    return None

Extracted PTPs are automatically pushed to CRM: the manager sees the promised date and amount; if unpaid, the system initiates a callback. Integration with 1C and Bitrix24 is included in the base package. To protect against hallucination, we set up validation via chain-of-thought prompts.

What Is Included in the Service

• Audit of the current collection process — analysis of scripts, compliance, and communication channels.
• Development of voice scripts — tailored to DPD stages, with A/B testing.
• Integration with CRM and PBX — REST API, webhook, SIP connection.
• Model training — fine-tuning GPT-4o-mini on historical call recordings.
• Pilot launch — 500–1000 debtors, 2 weeks, report with metrics.
• Documentation and support — API description, operator manual, SLA 8/5.

Process of Work

1. Analytics — we analyze your overdue portfolio, scripts, and infrastructure (1 week).
2. Design — architecture of scenarios, compliance rules, integration schemes (1 week).
3. Implementation — development of the voice bot, setup of NLP pipeline, connection to PBX (2–3 weeks).
4. Testing — QA on synthetic and real calls, adjustment of tone (1 week).
5. Deployment and monitoring — deployment on your servers or in the cloud, setup of logs and alerts (3 days).

Timeframes

Basic version (one scenario, one DPD slice) — 3–4 weeks. Full system with PTP tracking, multi-slice, and integration — about 2 months. Exact timing determined after audit.

Contact us for a free audit of your collection process. Order a pilot project — verify the effectiveness of AI calling on your base. We will evaluate the project within 2 business days.

Speech Recognition and Synthesis: ASR, TTS, Voice Cloning

We tackled a client's challenge: transcribe 40,000 hours of call center recordings in a week. Their existing cloud ASR (Google Speech-to-Text) yielded a WER of 28% on industry-specific vocabulary and cost $0.006 per minute — prohibitively expensive at that volume. The goal was to reduce WER below 10% and switch to self-hosted inference. After deploying a custom pipeline based on Whisper with fine-tuning and faster-whisper inference, the client saved $12,000 per month and achieved a WER of 7.3%.

How does speech recognition ASR handle noisy call center recordings?

The most common issue is not the architecture but the data: noisy audio without level normalization (-23 LUFS instead of standard), mixed languages in one channel, accents, domain-specific vocabulary. Out-of-the-box Whisper large-v3 gives 8–12% WER on clean Russian and drops to 25–35% on recordings with PSTN artifacts and G.711 narrowband codec. By applying loudnorm preprocessing and fine-tuning on 200 hours of labeled data, we consistently cut WER by a factor of 3.

Typical problems we encounter

WER does not converge to the desired metric. Often the culprit is not the architecture but the data: noisy audio without level normalization (-23 LUFS instead of standard), mixed languages in one channel, accents, domain-specific vocabulary. Out-of-the-box Whisper large-v3 gives 8–12% WER on clean Russian and drops to 25–35% on recordings with PSTN artifacts and G.711 narrowband codec.

Diarization fails with more than two speakers. pyannote/speaker-diarization-3.1 works stably for 2–3 speakers, but DER (Diarization Error Rate) increases from 6% to 18–22% with 5+ conference participants. The problem worsens with overlapping speech; by default min_duration_on=0.1 cuts short interjections. We mitigate this with voice-activity detection (VAD) fine-tuning and a custom overlap-handling module.

Voice cloning — latency vs. quality. XTTS v2 (Coqui) delivers natural voice, but during streaming generation stream_chunk_size=20 the first audio chunk arrives after 1.4–2.0 seconds — unacceptable for interactive scenarios. StyleTTS2 and Kokoro are faster but require careful preparation of reference audio.

How do we solve it in practice?

The basic stack for a production pipeline:

• ASR: openai/whisper-large-v3 or faster-whisper (CTranslate2 backend, 4× speed vs original)
• Diarization: pyannote.audio 3.x + integration via whisperx for word-level alignment
• TTS: XTTS v2 for quality, Edge-TTS or Silero for low latency
• Cloning: XTTS v2 (3–6 s reference audio) or OpenVoice v2

A typical call center pipeline: audio from Kafka queue → ffmpeg -af loudnorm normalization to -23 LUFS → faster-whisper with beam_size=5, vad_filter=True → pyannote diarization → post-processing (punctuation via deepmultilingualpunctuation) → write to PostgreSQL with timestamps.

Case study from our practice. A fintech company with 12,000 calls per day. Initial WER on Russian with banking vocabulary — 22% (Google STT). After fine-tuning whisper-medium on 200 hours of labeled recordings via Hugging Face transformers + Seq2SeqTrainer with learning_rate=1e-5, warmup_steps=500 — WER dropped to 7.3%. Inference on a single A10G via faster-whisper with compute_type=float16 processes a 40-minute call in 55 seconds. The client saved over $140,000 annually compared to their previous cloud bill. Contact us for a free pilot estimate to see similar savings on your data.

How to fine-tune Whisper on domain data?

When a general model underperforms, fine-tuning is the first tool. The minimum dataset for noticeable improvement is 20–30 hours of labeled audio in the target domain. Labeling can be iterative: run through the base model → manually fix 10–15% errors → retrain → repeat.

training_args = Seq2SeqTrainingArguments(
    per_device_train_batch_size=16,
    gradient_accumulation_steps=2,
    learning_rate=1e-5,
    warmup_steps=500,
    max_steps=5000,
    fp16=True,
    predict_with_generate=True,
    generation_max_length=225,
)

Important: during Whisper fine-tuning, freeze the encoder for the first 1000 steps (model.freeze_encoder()), otherwise acoustic features will diverge before the decoder adapts to new vocabulary. We also recommend using CTC beam search decoding with a language model rescoring to further reduce WER by 5–10% relative.

Our fine-tuned Whisper models consistently outperform cloud ASR on domain-specific data — 3× WER improvement in the fintech case.

Speech synthesis: How to choose a model for your task?

For interactive bots requiring TTFB < 300 ms — Silero or Kokoro. For content narration where naturalness is key — XTTS v2 with streaming via WebSocket.

Our process and deliverables

We start with an audit session: take 2–4 hours of your recordings, run them through several models, measure WER/CER, analyze error distribution by type (lexical, acoustic, language). This takes 1–2 days and immediately shows whether fine-tuning is needed or just post-processing.

Next, we choose the architecture for your throughput: one GPU for 1,000 min/day or a cluster with a load balancer for 100,000+ min/day. Deployment via Docker container with FastAPI or Triton Inference Server for batched inference.

What you get after engagement:

• Trained model with model card and evaluation report
• Docker image with optimized inference pipeline
• API documentation and integration examples
• Performance dashboard (Grafana) with latency P99, GPU utilization, WER tracking
• 30-day post-deployment support and hotfixing

Timelines depend on complexity:

• Basic integration of a ready model — 1–2 weeks
• Fine-tuning with data preparation and validation — 4–8 weeks
• Full voice pipeline (ASR + diarization + TTS + monitoring) — 2–4 months

Project investments typically range from $20,000 to $80,000. Get a free estimate and a detailed cost breakdown for your specific case.

Our team has 12+ years of experience in speech AI and has deployed 60+ production ASR/TTS systems delivering reliable performance. Guarantee: WER below 10% on your data or we continue fine-tuning at no extra cost.

Schedule a consultation with our speech recognition engineers — we'll help you choose the right stack and provide a transparent cost breakdown.