SAVA: Inworld's Voice Activity and End-of-Speech Detection Model

Uploaded by Inworld on 11 14 (same company as the Voice Isolator featured yesterday). SAVA processes 16 kHz mono real-time audio, outputting two probabilities every 40 ms: whether the current frame contains speech (with a 120 ms lookahead, resulting in a 120 ms delay) and whether the speaker has finished their turn (no lookahead). The shared network utilizes both acoustic and linguistic context to distinguish speech from coughs, laughter, music, and keyboard sounds. It has 25.5M parameters, featuring a backbone of causal log-mel features, 3 Mamba-2 blocks, and 1 attention blocks, followed by separate output heads for speech and end-of-speech detection. Officially, training used approximately 5,000 hours of real and synthetic conversational speech, mixed with non-speech vocalizations, noise, music, and room reverberation. Code and weights are licensed under Apache-2.0.

Original title: inworld/sava

Uploaded by Inworld on 11 14 (same company as the Voice Isolator featured yesterday). SAVA processes 16 kHz mono real-time audio, outputting two probabilities every 40 ms: whether the current frame contains speech (with a 120 ms lookahead, resulting in a 120 ms delay) and whether the speaker has finished their turn (no lookahead). The shared network utilizes both acoustic and linguistic context to distinguish speech from coughs, laughter, music, and keyboard sounds. It has 25.5M parameters, featuring a backbone of causal log-mel features, 3 Mamba-2 blocks, and 1 attention blocks, followed by separate output heads for speech and end-of-speech detection. Officially, training used approximately 5,000 hours of real and synthetic conversational speech, mixed with non-speech vocalizations, noise, music, and room reverberation. Code and weights are licensed under Apache-2.0.

Why it matters A small model performs two tasks simultaneously, runs on CPU, and is Apache-licensed; it can distinguish human speech from coughs, laughter, music, and keyboard typing

Rising score 38 · Likes 57 · Downloads 0

Reasons

  • Official benchmarks on an Apple M3 Max CPU: processing one audio stream takes 3.8 ms per 40 ms frame; also tested on NVIDIA Ampere, Hopper, and Blackwell architectures
  • Voice activity detection compared against Silero VAD, and end-of-speech detection compared against pipecat smart-turn v3.2, with AUC scores provided per dataset rather than a combined ranking; the official team notes that SAVA has a 120 ms lookahead, making latency comparisons unequal
  • The repository includes a reference turn-taking policy that segments the two probability curves into utterances; applications can substitute their own policies

Usage

  • pip install inworld-sava (Python 3.10 or higher), then run sava run recording.wav; add --turns_dir turns/ to output segmented utterances and --curves curves.json to save probability curves; for microphone input, install inworld-sava[mic] and run sava mic
  • Python: use sava.load("inworld/sava") to load the model (automatically downloads and verifies hashes on first use), sava.run to obtain curves, and sava.segment to segment audio using sava.Policy(); for streaming, use sava.Session(model), calling push() for each 16 kHz audio chunk and flush() at the end
  • Defaults to CPU, with --device to specify other devices; the GitHub repository inworld-ai/sava contains integration guides and full benchmark tables, and AGENTS.md can be provided to coding agents for automated setup

Risks

  • Responds to any human voice, without speaker identification; handling interruptions and backchannels like "uh-huh" requires custom dialogue strategies in the application
  • The official team notes that quiet, voice-like noise may cause false triggers (the ESC-50 test specifically evaluates this failure mode); benchmarks are self-reported by the official team
  • Training recipes, code, and data are not publicly available. This entry has not been independently verified

Open

Hugging Face Trending

Explore the latest AI Radar updates