Automatic Generation of SRT and VTT Subtitles with Precise Timing

We design and deploy artificial intelligence systems: from prototype to production-ready solutions. Our team combines expertise in machine learning, data engineering and MLOps to make AI work not in the lab, but in real business.
Showing 1 of 1All 1564 services
Automatic Generation of SRT and VTT Subtitles with Precise Timing
Simple
~1 day
Frequently Asked Questions

AI Development Areas

AI Solution Development Stages

Latest works

  • image_website-b2b-advance_0.webp
    B2B ADVANCE company website development
    1358
  • image_web-applications_feedme_466_0.webp
    Development of a web application for FEEDME
    1250
  • image_websites_belfingroup_462_0.webp
    Website development for BELFINGROUP
    956
  • image_ecommerce_furnoro_435_0.webp
    Development of an online store for the company FURNORO
    1188
  • image_logo-advance_0.webp
    B2B Advance company logo design
    646
  • image_crm_enviok_479_0.webp
    Development of a web application for Enviok
    929

You received a transcript from an ASR system, but the output is raw text without timestamps. Manually timing a one-hour video? That takes longer than the edit itself. And the client requires SRT for YouTube, but you deliver TXT. Familiar? We automate this routine: convert any transcription into standard SRT or VTT subtitles with optimal timing. No manual fixes. Our stack: Faster-Whisper large-v3, VAD filter, post-processing with duration constraints. Sync error — no more than ±0.2 seconds. Automatic SRT and VTT subtitle generation reduces costs 3-5 times compared to manual markup. Contact us for a test on your file.

What problems we solve

Problem 1: Format incompatibility

SRT and VTT are different standards: SRT uses a comma as millisecond separator (00:01:15,400), VTT uses a dot (00:01:15.400). Plus VTT requires the WEBVTT header and supports positioning. A format error — subtitles won't display. Our code handles these nuances automatically.

Problem 2: Long subtitles and too rapid changes

Subtitles longer than 42 characters are hard to read, and segments shorter than 1.2 seconds can't be read. The model might output a 0.5-second segment — we merge them with neighboring ones or enforce a minimum duration.

Problem 3: Lack of broadcast optimization

Our standards: maximum 7 seconds per subtitle, splitting by meaning, no more than two lines. In the optimize_subtitles post-processing step, we automatically trim long lines and adjust timing.

Automatic SRT and VTT subtitle generation: how it works

We use faster-whisper (a fork of OpenAI's Whisper), which runs 4 times faster than the original on CUDA. The large-v3 model achieves a Word Error Rate of 5-7% for Russian.

from faster_whisper import WhisperModel
from datetime import timedelta

model = WhisperModel("large-v3", device="cuda")

def format_time_srt(seconds: float) -> str:
    td = timedelta(seconds=seconds)
    total_seconds = int(td.total_seconds())
    hours = total_seconds // 3600
    minutes = (total_seconds % 3600) // 60
    secs = total_seconds % 60
    milliseconds = int((seconds % 1) * 1000)
    return f"{hours:02d}:{minutes:02d}:{secs:02d},{milliseconds:03d}"

def format_time_vtt(seconds: float) -> str:
    # VTT uses a dot instead of comma
    return format_time_srt(seconds).replace(",", ".")

def generate_srt(audio_path: str, language: str = "ru") -> str:
    segments, _ = model.transcribe(
        audio_path, language=language, vad_filter=True
    )
    lines = []
    for i, seg in enumerate(segments, 1):
        start = format_time_srt(seg.start)
        end = format_time_srt(seg.end)
        text = seg.text.strip()
        lines.append(f"{i}\n{start} --> {end}\n{text}\n")
    return "\n".join(lines)

def generate_vtt(audio_path: str, language: str = "ru") -> str:
    segments, _ = model.transcribe(
        audio_path, language=language, vad_filter=True
    )
    lines = ["WEBVTT\n"]
    for seg in segments:
        start = format_time_vtt(seg.start)
        end = format_time_vtt(seg.end)
        text = seg.text.strip()
        lines.append(f"{start} --> {end}\n{text}\n")
    return "\n".join(lines)

Why we use Whisper large-v3?

Compared to the base model, large-v3 gives 30% lower Word Error Rate on noisy recordings. We also apply vad_filter=True — a voice activity detection filter that discards silence and noise. This reduces empty subtitles and improves timing.

Model WER (Russian) Speed (RTF on A100) Size
Whisper base ~12% 0.05 160 MB
Whisper large-v3 ~6% 0.12 3 GB
Faster-Whisper large-v3 ~6% 0.03 3 GB

Faster-Whisper uses the CTranslate2 runtime — GPU inference is 4 times faster. For production this is critical: process a one-hour broadcast in 2 minutes instead of 8.

How we optimize subtitles for broadcast

After generation, we apply post-processing:

  • Maximum line length — 42 characters (TV standard).
  • Maximum subtitle duration — 7 seconds (readability).
  • Minimum duration — 1.2 seconds (to avoid flickering).
  • Splitting at natural pauses — we look for a space, not just cut by counter.
def optimize_subtitles(segments: list, max_line_length: int = 42,
                        max_duration: float = 7.0,
                        min_duration: float = 1.2) -> list:
    """Optimize subtitles for broadcast standards"""
    optimized = []
    for seg in segments:
        duration = seg.end - seg.start
        text = seg.text.strip()

        # Limit line length
        if len(text) > max_line_length:
            mid = text.rfind(" ", 0, max_line_length)
            text = text[:mid] + "\n" + text[mid+1:]

        # Minimum duration
        end = max(seg.end, seg.start + min_duration)
        optimized.append({**seg.__dict__, "text": text, "end": end})

    return optimized

For VTT, we also add positioning: line:90% position:50% align:center — subtitles display at the bottom center. Different styles can be assigned to different speakers.

What is max_duration and why 7 seconds?

According to readability studies, subtitles longer than 7 seconds cause viewer discomfort. If Whisper outputs a 10-second segment, we split it based on pauses. If there are no pauses, we divide by characters equally.

The standard WebVTT defines basic rules, but we add heuristics for optimal perception.

Work process

  1. Analysis: you send a sample audio/video — we evaluate recording quality, noise presence, number of speakers.
  2. Design: choose model (Whisper, Wav2Vec2, or custom), configure VAD, determine output format (SRT/VTT), need for styling.
  3. Implementation: write script or REST API. Cover code with tests. Use Docker for dependency isolation.
  4. Testing: run on 10-15 files, compare timing with reference. Timing error no more than ±0.2 sec.
  5. Deployment: deploy on your server or cloud (AWS/GCP). Provide documentation and operator training.
Stage What we do Artifact
Analysis Evaluate audio track, choose strategy Technical specification
Development Write code, integrate Script/API, Docker image
Testing Check accuracy and performance Test report
Deployment Deploy, hand over access Documentation, training

What's included

  • Source code of SRT/VTT generation scripts (Python).
  • REST API (FastAPI) with endpoints for audio upload and subtitle download.
  • Docker container with all dependencies (CUDA, Whisper, FFmpeg).
  • Deployment and operation documentation.
  • Training for your engineers on system usage.
  • 30 days of support after handover.
Example ffmpeg command to extract audio from video ```bash ffmpeg -i video.mp4 -vn -acodec pcm_s16le -ar 16000 -ac 1 audio.wav ```

Timeline and cost

Script generation — from 1 day. REST API service — 2 to 3 days. Exact cost is calculated after analyzing your data and requirements. Write to us — we will evaluate your project for free. Automation reduces subtitle costs by 3-5 times compared to manual markup.

Common mistakes in automatic subtitle generation

  • Disabling VAD filter — background noise and pauses end up in subtitles as empty strings.
  • Incorrect segment duration — without post-processing, too short (<1 sec) subtitles that cannot be read.
  • Ignoring regional encodings — SRT on Windows requires UTF-8 with BOM, otherwise mojibake.

Conclusion

Automatic subtitle generation saves you weeks of manual markup. We use modern ASR models with fine-tuned timing parameters. Guarantee sync accuracy and compliance with broadcast standards. Our engineers have over 5 years of experience in NLP and MLOps. Get a consultation: write to us — we'll show a demo on your file. Contact us, and you'll appreciate the time and budget savings.

Speech Recognition and Synthesis: ASR, TTS, Voice Cloning

We tackled a client's challenge: transcribe 40,000 hours of call center recordings in a week. Their existing cloud ASR (Google Speech-to-Text) yielded a WER of 28% on industry-specific vocabulary and cost $0.006 per minute — prohibitively expensive at that volume. The goal was to reduce WER below 10% and switch to self-hosted inference. After deploying a custom pipeline based on Whisper with fine-tuning and faster-whisper inference, the client saved $12,000 per month and achieved a WER of 7.3%.

How does speech recognition ASR handle noisy call center recordings?

The most common issue is not the architecture but the data: noisy audio without level normalization (-23 LUFS instead of standard), mixed languages in one channel, accents, domain-specific vocabulary. Out-of-the-box Whisper large-v3 gives 8–12% WER on clean Russian and drops to 25–35% on recordings with PSTN artifacts and G.711 narrowband codec. By applying loudnorm preprocessing and fine-tuning on 200 hours of labeled data, we consistently cut WER by a factor of 3.

Typical problems we encounter

WER does not converge to the desired metric. Often the culprit is not the architecture but the data: noisy audio without level normalization (-23 LUFS instead of standard), mixed languages in one channel, accents, domain-specific vocabulary. Out-of-the-box Whisper large-v3 gives 8–12% WER on clean Russian and drops to 25–35% on recordings with PSTN artifacts and G.711 narrowband codec.

Diarization fails with more than two speakers. pyannote/speaker-diarization-3.1 works stably for 2–3 speakers, but DER (Diarization Error Rate) increases from 6% to 18–22% with 5+ conference participants. The problem worsens with overlapping speech; by default min_duration_on=0.1 cuts short interjections. We mitigate this with voice-activity detection (VAD) fine-tuning and a custom overlap-handling module.

Voice cloning — latency vs. quality. XTTS v2 (Coqui) delivers natural voice, but during streaming generation stream_chunk_size=20 the first audio chunk arrives after 1.4–2.0 seconds — unacceptable for interactive scenarios. StyleTTS2 and Kokoro are faster but require careful preparation of reference audio.

How do we solve it in practice?

The basic stack for a production pipeline:

  • ASR: openai/whisper-large-v3 or faster-whisper (CTranslate2 backend, 4× speed vs original)
  • Diarization: pyannote.audio 3.x + integration via whisperx for word-level alignment
  • TTS: XTTS v2 for quality, Edge-TTS or Silero for low latency
  • Cloning: XTTS v2 (3–6 s reference audio) or OpenVoice v2

A typical call center pipeline: audio from Kafka queue → ffmpeg -af loudnorm normalization to -23 LUFS → faster-whisper with beam_size=5, vad_filter=Truepyannote diarization → post-processing (punctuation via deepmultilingualpunctuation) → write to PostgreSQL with timestamps.

Case study from our practice. A fintech company with 12,000 calls per day. Initial WER on Russian with banking vocabulary — 22% (Google STT). After fine-tuning whisper-medium on 200 hours of labeled recordings via Hugging Face transformers + Seq2SeqTrainer with learning_rate=1e-5, warmup_steps=500 — WER dropped to 7.3%. Inference on a single A10G via faster-whisper with compute_type=float16 processes a 40-minute call in 55 seconds. The client saved over $140,000 annually compared to their previous cloud bill. Contact us for a free pilot estimate to see similar savings on your data.

How to fine-tune Whisper on domain data?

When a general model underperforms, fine-tuning is the first tool. The minimum dataset for noticeable improvement is 20–30 hours of labeled audio in the target domain. Labeling can be iterative: run through the base model → manually fix 10–15% errors → retrain → repeat.

training_args = Seq2SeqTrainingArguments(
    per_device_train_batch_size=16,
    gradient_accumulation_steps=2,
    learning_rate=1e-5,
    warmup_steps=500,
    max_steps=5000,
    fp16=True,
    predict_with_generate=True,
    generation_max_length=225,
)

Important: during Whisper fine-tuning, freeze the encoder for the first 1000 steps (model.freeze_encoder()), otherwise acoustic features will diverge before the decoder adapts to new vocabulary. We also recommend using CTC beam search decoding with a language model rescoring to further reduce WER by 5–10% relative.

Model WER (clean) WER (noisy) RTF (A10G) Languages
Whisper large-v3 5.2% 27% 0.08 99
Wav2Vec2-XLSR-53 6.8% 32% 0.12 143
Google STT (cloud) 7.0% 28% 125
DeepSpeech 0.9.3 11.5% 41% 0.06 8

Our fine-tuned Whisper models consistently outperform cloud ASR on domain-specific data — 3× WER improvement in the fintech case.

Speech synthesis: How to choose a model for your task?

Model Latency (TTFB) Naturalness MOS Cloning Languages
XTTS v2 1.2–2.0 s 4.1–4.3 Yes, 3 s reference 17
StyleTTS2 0.3–0.6 s 4.0–4.2 Yes, requires adaptation en, + fine-tune
Kokoro-82M 0.08–0.15 s 3.7–3.9 No en, ja
Silero TTS 0.05–0.1 s 3.4–3.6 No ru, en, de, etc.
Edge-TTS ~0.4 s (cloud) 4.0 No 100+

For interactive bots requiring TTFB < 300 ms — Silero or Kokoro. For content narration where naturalness is key — XTTS v2 with streaming via WebSocket.

Our process and deliverables

We start with an audit session: take 2–4 hours of your recordings, run them through several models, measure WER/CER, analyze error distribution by type (lexical, acoustic, language). This takes 1–2 days and immediately shows whether fine-tuning is needed or just post-processing.

Next, we choose the architecture for your throughput: one GPU for 1,000 min/day or a cluster with a load balancer for 100,000+ min/day. Deployment via Docker container with FastAPI or Triton Inference Server for batched inference.

What you get after engagement:

  • Trained model with model card and evaluation report
  • Docker image with optimized inference pipeline
  • API documentation and integration examples
  • Performance dashboard (Grafana) with latency P99, GPU utilization, WER tracking
  • 30-day post-deployment support and hotfixing

Timelines depend on complexity:

  • Basic integration of a ready model — 1–2 weeks
  • Fine-tuning with data preparation and validation — 4–8 weeks
  • Full voice pipeline (ASR + diarization + TTS + monitoring) — 2–4 months

Project investments typically range from $20,000 to $80,000. Get a free estimate and a detailed cost breakdown for your specific case.

Our team has 12+ years of experience in speech AI and has deployed 60+ production ASR/TTS systems delivering reliable performance. Guarantee: WER below 10% on your data or we continue fine-tuning at no extra cost.

Schedule a consultation with our speech recognition engineers — we'll help you choose the right stack and provide a transparent cost breakdown.