Original title: inworld/voice-isolator
Vocal separation model v1 uploaded by Inworld on 11 14, with two checkpoints sharing the same architecture: the offline version voice-isolator-v1 (64M parameters, each frame sees the entire 8 seconds) for recordings, and the real-time version voice-isolator-v1-realtime (58M parameters, causal streaming, with a lookahead of 20 ms resulting in an algorithmic latency of 63 ms) for live streaming and calls. Input is 48 kHz mono, output consists of two streams: vocals and the removed background; summing the two streams reconstructs the original input. v1 only performs noise reduction and does not restore frequency bands missing from the recording (restoration features are planned for v2). Both code and weights are licensed under Apache-2.0.
Why it matters Apache license allows commercial use; the real-time version has low enough latency for calls and live streaming; it can clean up recordings and organize voice data, and the background residual can also be used for accompaniment separation
Rising score 56 · Likes 60 · Downloads 0
Reasons
- The architecture is a dual-path network operating on sub-bands of the STFT: each frame is split into 46 sub-bands, with 4 attention blocks and 3 Mamba blocks, outputting bounded complex masks and residuals
- Officially, training used approximately 10,000 hours of audio, covering multilingual reading and conversation, character dubbing, whispering, and singing, while preserving the speaker's own laughter, breathing, and sighs
- When two people speak simultaneously, it retains the louder one; when loudness is similar, it typically retains both; it does not select based on a specified speaker
Usage
- pip install inworld-voice-isolation, then voice-isolate in.wav out.wav (the offline model is automatically downloaded and hash-verified on first use); the removed background is saved separately as out_background.wav; --no-background saves only the vocals, --output-sr 0 maintains the original sample rate
- Streaming: VoiceIsolator("inworld/voice-isolator", mode="streaming", lookahead_frames=2) followed by calling stream(), pushing 960 samples every 20 ms, and calling flush() at the end; add --mode streaming --lookahead 2 in the command line
- Use CUDA if available, mps for Apple Silicon Macs (PyTorch 2.6 or higher), otherwise use CPU; GitHub inworld-ai/voice-isolator includes an interactive demo and benchmark documentation, and AGENTS.md can be provided to coding agents for installation
Risks
- Officially tested only on A100, H100, B200, B300, and CPU; smaller GPUs have not been tested
- The VIBE benchmark score chart in the card comes from Inworld's own internal evaluation set, which is not public
- Only noise reduction, no restoration: frequency bands lost in the recording itself will not be restored. This entry has not been run