Development of an AI Transcription System with API
You have a call analytics startup, and you integrated a ready-made transcription API — within a month the cost per minute of audio tripled, and the p99 latency exceeded 12 seconds. A typical situation: third-party services don't scale to your volume, and their pricing is uncontrollable. We build private transcription APIs — with full control over the model, infrastructure, and cost. With experience in ML and 40+ deployed systems, we guarantee stable performance under any load.
Why Deploy Your Own Transcription API?
Off-the-shelf solutions fall short when you need to process hundreds of hours of audio per day, achieve sub-5-second latency, or customize the model for your domain. A custom API built on Whisper with diarization and batching delivers p99 < 4 seconds even on an A10G. The operational cost per minute is 5–10 times lower than cloud providers. Our API is 4 times faster than cloud counterparts in p99 for transcribing one minute of audio.
What Problems We Solve
Latency. During audio calls, the client waits for the transcript to search for keywords. Our API streams results via WebSocket with < 3 seconds delay on a 30-second chunk. For comparison, cloud APIs typically have p99 of 8–15 seconds — at least 4 times longer.
Diarization quality. Standard APIs confuse speakers if voice timbres are similar. We use pyannote-audio with a pre-trained embedder — diarization accuracy of 92% on the CorpSet.
Billing integration. You need to charge for each minute of transcription. The API includes a built-in consumption counter with automatic limits and notifications.
How We Do It
Stack and Architecture
- Model:
Whisper Large V3(INT8 quantized) inferred via TensorRT — up to 2x speedup on the same GPU. - Service:
FastAPIasynchronously (Uvicorn + Gunicorn). Task queue on Celery with Redis. For streaming — WebSocket via WebSockets. - Diarization:
pyannote-audioin a separate container, results merged by timestamps. - Batching:
vLLMfor Whisper — groups up to 32 audio chunks into one batch, smoothing latency.
Real-World Case
A call center analytics platform was processing 4000 hours of audio per month via a cloud API — after deployment, the budget was cut 5 times. We deployed a private API on 2× L40S: p99 dropped from 15 s to 3.2 s. Stack: Whisper + pyannote + TensorRT + Celery. We implemented batching and result caching for repeated requests.
How We Ensure High Availability
The architecture is built on Kubernetes with automatic horizontal scaling: as load increases, inference pods are added. Monitoring via Prometheus + Grafana, alerts on p99 latency and GPU utilization. 99.9% uptime is contractually guaranteed.
Process of Work
- Analysis: We break down your requirements — volume, latency, model, CRM integration.
- Design: API specification (REST + WebSocket), inference server choice (vLLM / TGI / Triton), billing scheme.
- Development: Implementation of endpoints, diarization, batching, SDK (Python/JS).
- Testing: Load testing with your data — measuring p99, FLOPS, GPU utilization.
- Deployment: Containerization (Docker + Kubernetes), monitoring (Prometheus + Grafana), documentation (OpenAPI + Postman).
What's Included
- REST API + WebSocket endpoints (as per specification above)
- Webhook notifications for task status
- SDK for Python and JavaScript
- Per-minute billing with limits
- OpenAPI documentation + Swagger UI
- 1 month of support after deployment
Comparison of Approaches
| Parameter | Ready-Made Cloud API | Custom API (Our Development) |
|---|---|---|
| latency p99 (1 min audio) | 8–15 s | 2–4 s |
| Cost / minute | high (depends on provider) | low (depends on load) |
| Model customization | no | full (fine-tuning, LoRA) |
| Data control | under NDA | your infrastructure |
| Scaling | quotas | automatic horizontal scaling |
Comparison of Inference Models
| Model | Speed (latency p99) | Quality (WER) | GPU Memory |
|---|---|---|---|
| Whisper Large V3 (FP16) | 6.1 s | 8.2% | 10 GB VRAM |
| Whisper Large V3 (INT8) | 3.4 s | 8.5% | 5 GB VRAM |
| Distil-Whisper (FP16) | 1.8 s | 10.1% | 4 GB VRAM |
How We Optimize the Model for Your Data
If your corpus contains specialized vocabulary (medical, legal), we can fine-tune Whisper using LoRA. This reduces WER by an additional 5–15% without increasing latency. A labeled dataset of at least 10 hours is required.
Estimated Timeline
- Baseline API (REST + WebSocket, single model, diarization): 2 to 4 weeks.
- With billing, SDK, load testing: 1 to 2 months.
- From specification approval, depending on customization complexity.
We'll provide an accurate estimate after analyzing your requirements — contact us to discuss details within 1 day. Over 40 companies have entrusted us with their transcription systems. Request a free consultation with no obligation.
Original diarization research: pyannote-audio