Original title: salesforce-sovereign-intelligence/SOV-Transcribe-1
Non-streaming speech recognition model uploaded by Salesforce on 11 10, with the model card stating a total of 26.6B parameters (4.6B active, including the audio encoder), composed of a Whisper large-v3 audio encoder, a projection layer, and a Gemma 4 26B A4B MoE. Supports 7 languages: English, French, German, Spanish, Italian, Portuguese, and Dutch; tested with single audio segments up to 30 minutes. Natural language instructions can be used to select between verbatim or cleaned transcripts, control punctuation and number formatting, and provide names, terms, or topic descriptions to help recognize difficult words. Apache-2.0 license.
Why it matters Can be deployed on your own servers; recognizes details like names, phone numbers, and email addresses more accurately; controls whether to produce verbatim or cleaned transcripts with a single sentence
Rising score 46 · Likes 60 · Downloads 572
Reasons
- Officially stated that as of 11 12, it ranks 1 among open-weight models (and 7 overall) on the Artificial Analysis non-streaming speech-to-text leaderboard, and ranks 2 in entity accuracy and task success rate among 23 systems on VoiceCodeBench
- Includes a DFlash speculative decoding draft model, speeding up decoding by approximately 2–3 times for single requests; officially measured on one RTX PRO 6000 Blackwell GPU that 30 seconds of audio returns in 0.22 seconds, transcribing up to 1,039 seconds of audio per second
- The model card provides prompt templates (verbatim, keywords, auto language detection, specified language) as well as prompts and scoring scripts for various evaluations, facilitating reproducibility
Usage
- Transformers (5.19 <= version < 5.23): Load using AutoProcessor and AutoModelForMultimodalLM with trust_remote_code=True; read audio at 16 kHz mono; call processor.apply_transcription_request(audio, prompt="...") followed by generate
- vLLM (officially recommended): Download the vllm_plugin/* files from this repository via hf download and install them with pip (which installs vLLM 0.32.0), then run vllm serve using the OpenAI-compatible /v1/audio/transcriptions endpoint; default duration is 600 seconds per request, set VLLM_MAX_AUDIO_DECODE_DURATION_S to 1800 for 30 minutes
- For proper noun hints, use: This audio may contain the following words or phrases: {keywords}. Transcribe the audio into text.; officially recommends greedy decoding (temperature=0) and disabling thinking
Risks
- Does not support Chinese or streaming recognition; rankings and speed claims are official statements, with evaluation tables provided only as images
- Native support is still being merged into vLLM and Transformers, currently requiring trust_remote_code or the official plugin; speed was measured on a professional GPU with 96 GB of VRAM, and it is not specified whether standard GPUs can accommodate it
- In rare cases, it may repeat a character or phrase until reaching the length limit, which the official team removes via post-processing; for long recordings, segmenting by pauses is recommended. This entry has not been run