We often encounter a situation: a client needs audio transcription for hundreds of hours of audio monthly. Cloud APIs are either expensive or insecure—data leaves your premises and costs scale linearly. Self-hosted Whisper gives you full control over data, predictable costs at high volumes, and the ability to fine-tune for a specific accent or domain. For example, in one project we deployed Whisper large-v3 on two A10G GPUs, processing up to 8 hours of audio per hour with accuracy comparable to cloud solutions, but at more than 4x cost savings. We used a VAD filter and word_timestamps for subtitle synchronization. This configuration can handle up to 2000 hours of audio per month on a single GPU server. To estimate your load, contact our engineer—we will find the optimal configuration.
Solving common problems
- Low accuracy on noisy audio: VAD filter and beam_size tuning improve recognition. We adjust parameters for your audio type.
- High latency for streaming recording: We use chunking and WebSocket.
- Lack of monitoring: Prometheus + Grafana track GPU utilization and queue depth.
Cost savings with self-hosted Whisper
When transcribing more than 3000 minutes per month, a dedicated server pays for itself faster. Cloud pricing is linear, while self-hosted on an A10G at 50% load saves 3–6 times. For example, at 3000 minutes/month, cloud costs approximately $180 (at $0.06/min) vs self-hosted around $60/month for GPU rental. For 10,000 minutes, cloud costs $600 vs self-hosted $150, saving $450 monthly. For 50,000 minutes, cloud costs $3,000 vs self-hosted $750, saving $2,250. Moreover, you have full data control and can customize the model for your domain. Reducing transcription costs directly improves ROI. Get a consultation—we will calculate the savings for your volume.
Hardware requirements by volume
Whisper hardware requirements vary by load. Whisper requires a discrete NVIDIA GPU with CUDA support. Recommended configurations:
| Load | GPU | RAM | Disk |
|---|---|---|---|
| Up to 10 hours/day | RTX 3080 10GB | 16 GB | 100 GB SSD |
| Up to 100 hours/day | RTX 4090 | 32 GB | 500 GB SSD |
| More than 100 hours/day | 2x A10G | 64 GB | 2 TB NVMe |
For GPU transcription, we recommend at least an RTX 3080.
Production deployment architecture
Production deployment includes several key components, integrating Celery Whisper task queue for robust job management:
Audio Input → Nginx → FastAPI Workers → Whisper Workers (GPU) → PostgreSQL
↓ ↓
Redis Queue S3 Storage
Main components:
- FastAPI — REST API for receiving tasks
- Celery — asynchronous processing queue; we use Celery for managing Whisper tasks, with retry and monitoring
- Redis — task broker and cache
- faster-whisper — inference engine (CTranslate2), optimized using int8_float16 quantization and beam search decoding
- PostgreSQL — storing transcriptions and metadata
Setup guide
- Install Docker and NVIDIA Container Toolkit.
- Build the worker image with faster-whisper and dependencies.
- Launch Redis and PostgreSQL.
- Deploy the FastAPI application implementing REST endpoints.
- Run the Celery worker with GPU binding via
--gpus all. - Configure monitoring: Whisper monitoring is achieved via Prometheus and Grafana, tracking queue depth and GPU utilization.
- Test with sample audio files, varying language and duration.
Worker configuration
Celery worker configuration for faster-whisper with retry and monitoring, employing temperature scheduling for improved accuracy:
from celery import Celery
from faster_whisper import WhisperModel
app = Celery('whisper_tasks', broker='redis://localhost:6379/0')
model = WhisperModel("large-v3", device="cuda", compute_type="int8_float16")
@app.task(bind=True, max_retries=3)
def transcribe_audio(self, file_path: str, language: str = None):
try:
segments, info = model.transcribe(
file_path,
language=language,
vad_filter=True,
word_timestamps=True
)
return {
"language": info.language,
"duration": info.duration,
"segments": [
{"start": s.start, "end": s.end, "text": s.text}
for s in segments
]
}
except Exception as exc:
raise self.retry(exc=exc, countdown=60)
Model selection
Model choice affects accuracy and speed. In production, large-v3 is most common, but for lightweight tasks medium suffices. Comparison based on faster-whisper data (faster-whisper GitHub repository):
| Model | VRAM | Speed (xRT) | WER (English) |
|---|---|---|---|
| tiny | ~1 GB | ~32x | ~7.7% |
| base | ~1 GB | ~16x | ~5.2% |
| small | ~2 GB | ~6x | ~4.0% |
| medium | ~5 GB | ~2x | ~3.0% |
| large-v3 | ~10 GB | ~1x | ~2.2% |
*Speed relative to real-time (higher xRT = faster). For example, the tiny model is 32 times faster than real-time, making it 5 times faster than medium and 32 times faster than large-v3.
Monitoring and reliability
- Celery Flower for task queue monitoring
- Prometheus + Grafana for GPU utilization and queue depth metrics
- Automatic worker restart via systemd
- Healthcheck endpoint checking GPU availability
Example docker-compose.yml for deployment
version: '3.8'
services:
redis:
image: redis:7
db:
image: postgres:15
api:
build: ./api
depends_on: [redis, db]
worker:
build: ./worker
deploy:
resources:
reservations:
devices:
- capabilities: [gpu]
Economic advantages of self-hosted Whisper
OpenAI Whisper server can be self-hosted for privacy and cost savings. When transcribing more than 3000 minutes per month, a dedicated server pays for itself faster. Cloud pricing is linear, while self-hosted on an A10G at 50% load saves 3–6 times. For example, at 3000 minutes/month, cloud costs approximately $180 (at $0.06/min) vs self-hosted around $60/month for GPU rental. Moreover, you have full data control and can customize the model for your domain. Reducing transcription costs directly improves ROI. Get a consultation—we will calculate the savings for your volume.
What the work includes
- Audit of audio load and GPU configuration selection.
- Deployment of FastAPI + Celery + Redis + PostgreSQL.
- Configuration of faster-whisper with VAD filter and word_timestamps.
- Integration with S3-compatible storage.
- Monitoring with Prometheus + Grafana.
- Deliverables: API documentation, Git repo with Docker Compose files, API keys for S3, monitoring dashboard access, 1-hour team training session, 2 weeks post-deployment support.
Timeline and cost
- Basic deployment: 2–3 days.
- With task queue and API: 5–7 days.
- Full production system with monitoring: up to 2 weeks.
- Cost is calculated individually based on your load and requirements. Setup starts at $2,000 for basic deployment.
Our experience with Whisper deployments: over 30 projects. We guarantee stable operation and timely support. If you are interested in implementing self-hosted Whisper, get an engineer consultation—we will prepare a proposal and evaluate your project within a day.