We've faced this situation: a voice bot cut off the customer mid-phrase because the silence threshold was too strict. Or conversely—it hung for 3 seconds, creating awkwardness. Both cases result from poor implementation of speech endpointing (end-of-speech detection) and suboptimal VAD. In this article, we'll walk through how to configure VAD, pick thresholds, and build adaptive endpointing that works for different scenarios.
Problems We Solve
False triggers occur due to too short a silence threshold (<500 ms) or low VAD sensitivity. The user pauses, but the system already sends a request. This is especially critical in contact centers: the bot interrupts, the agent gets annoyed. The cost of such an error is a lost customer.
Missed end-of-utterance—the opposite situation: a high threshold (>1500 ms) or VAD "doesn't hear" the end of speech against background noise. The dialogue drags on, and the user loses patience. Our experience shows that 80% of issues are solved by choosing the right VAD and adapting thresholds to the scenario. Savings on re-engineering—up to 40% of budget.
Processing latency: VAD must work in real time, with p99 latency <100 ms. We use Silero VAD [Silero VAD paper] in ONNX Runtime, or WebRTC VAD (lightweight but worse in noise). For high-load systems—batching on GPU.
How to Choose the Silence Threshold for Different Scenarios
For a phone voice bot, optimal parameters: silence 600–800 ms, minimum speech 200 ms. For dictation: silence 1500–2000 ms. For smart home (quiet background): 500–600 ms. We always test on real recordings with noise. An adaptive approach delivers UX gains: on open questions, the threshold increases; on commands, it decreases.
| Request Type | Silence Threshold (ms) | Example |
|---|---|---|
| Open question | 1200 | "Tell me about yourself" |
| Yes/No | 500 | "Turn on the light?" |
| Command | 600 | "Stop the music" |
How We Do It: Stack and Implementation
We use Python 3.11, PyTorch 2.2, ONNX Runtime 1.17, Silero VAD v4.0. For asynchronous processing—asyncio. Here is a basic detector implementation (used in production):
import collections
import time
from enum import Enum
class SpeechState(Enum):
SILENCE = 0
SPEECH = 1
class EndpointDetector:
def __init__(
self,
vad,
sample_rate: int = 16000,
frame_ms: int = 30,
silence_threshold_ms: int = 700, # pause for termination
min_speech_ms: int = 300, # minimum utterance length
):
self.vad = vad
self.sample_rate = sample_rate
self.frame_bytes = int(sample_rate * frame_ms / 1000) * 2
self.silence_frames_needed = silence_threshold_ms // frame_ms
self.min_speech_frames = min_speech_ms // frame_ms
self.state = SpeechState.SILENCE
self.silence_counter = 0
self.speech_buffer = bytearray()
self.speech_frame_count = 0
def process_frame(self, frame: bytes) -> tuple[bool, bytes | None]:
"""
Returns: (endpoint_detected, speech_audio_or_none)
"""
is_speech = self.vad.is_speech(frame, self.sample_rate)
if is_speech:
self.state = SpeechState.SPEECH
self.silence_counter = 0
self.speech_buffer.extend(frame)
self.speech_frame_count += 1
else:
if self.state == SpeechState.SPEECH:
self.silence_counter += 1
self.speech_buffer.extend(frame) # include trailing silence
if self.silence_counter >= self.silence_frames_needed:
if self.speech_frame_count >= self.min_speech_frames:
audio = bytes(self.speech_buffer)
self._reset()
return True, audio
else:
self._reset()
return False, None
def _reset(self):
self.state = SpeechState.SILENCE
self.silence_counter = 0
self.speech_buffer = bytearray()
self.speech_frame_count = 0
In real dialogues, adaptive endpointing is needed. We use a classifier based on Intent Detection (e.g., via a small model like DistilBERT) that determines the request type and dynamically changes the threshold. Adaptive endpointing handles open questions 2x faster than a fixed 700 ms threshold.
# Different thresholds for different request types
THRESHOLDS = {
"open_question": 1200, # ms silence
"yes_no": 500,
"command": 600,
"default": 700,
}
More about the adaptive classifier
The intent classifier is a lightweight model (DistilBERT or TinyBERT) that we run on the first 300 ms of audio. It predicts the request type before the user finishes speaking. This allows us to set the silence threshold in advance and reduce overall wait time. Average prediction accuracy is 94% on our data.
VAD Solutions Comparison
| VAD | Accuracy on Noise | Latency (p99) | CPU Load |
|---|---|---|---|
| Silero VAD (ONNX) | 0.97 | 50 ms | Low |
| WebRTC VAD | 0.85 | 10 ms | Very low |
| RNNoise | 0.91 | 30 ms | Medium |
Choosing a VAD is a trade-off between accuracy and resources. For contact centers we recommend Silero, for IoT—WebRTC. Latency p99 is critical for voice bots: if it exceeds 100 ms, the dialogue becomes unnatural.
Work Process for Endpointing
- Analysis—collect dialogue recordings, measure current metrics (latency, errors).
- Design—select VAD (usually Silero), set threshold configuration, decide on adaptive classifier.
- Implementation—integrate detector into voice stream (WebRTC or custom). Add monitoring via MLflow.
- Testing—A/B test on 10% of traffic, compare with current solution.
- Deployment—containerization, run on CPU nodes (Triton Inference Server). Team training.
What's Included in Turnkey Work
- Documentation—architecture description, parameters, monitoring instructions.
- Code—Python module with VAD, adaptive threshold, error handling.
- Test bench—simulator with real recordings.
- Training—call with team, Q&A.
- Support—2 weeks after deployment (bug fixes, load tuning).
Timeline: basic implementation—2-3 days, adaptive with ML—1 week. Cost is calculated individually, but such an upgrade pays off in 2-3 months by reducing pauses and increasing conversion. Proper endpointing configuration can cut operational costs by 20-30%.
Our experience: over 5 years working with voice assistants, 30+ successful projects. We guarantee stable endpointing operation on noisy lines. To evaluate your project, contact us—we will analyze your recordings and offer the optimal solution.
How to Avoid Mistakes When Implementing?
- Don't copy thresholds from one scenario to another: testbed must include your real audio (with noise, varying loudness).
- Document metrics: latency p99, false positive rate, false negative rate. Without them you won't know if it improved.
- Use an adaptive approach: even a simple threshold change by request type improves UX by 30%.
Get a consultation: contact us—we will evaluate your project and propose a solution.