Original title: nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B
NVIDIA's offline Speech/Audio Large Model (SALM): A Transformer ASR encoder and the Nemotron-3-Diarization speaker encoder feed audio into the Nemotron-3.5 Lightning 30B-A3B hybrid Mamba-Transformer MoE language model, with a total of 34B parameters and approximately 4.3B active parameters (including the audio encoder). It supports transcription in 40 languages, handles recordings up to 1 hours long (with a context window of 65,536 tokens), and allows users to select transcription styles via instructions (verbatim including filler words, cleaned up for readability, incorporating domain terminology, or identifying who said what and when). It also supports speech translation, language identification, and audio question answering, with optional word-level timestamps. NVIDIA has also released FP8, NVFP4 versions, and a DFlash2 speculative decoding draft model. The card lists the release date as 10 28, which was not included in previous issues.
Rising score 334.5 · Likes 3057 · Downloads 57297
Reasons
- NVIDIA claims an average cpWER of 15.80 for multi-speaker full-meeting transcription (AMI, CALLHOME, Mixer 6, NOTSOFAR-1), which is lower than the previous best public results of 16.83. Among open-weight models, it achieves the lowest scores in the table for the English Open ASR leaderboard (average WER of 4.71 across 9 public sets), long-form audio (10.98), and the multilingual leaderboard (4.62 across 7 languages) (results as of 10 12).
- A single model replaces multiple tools for transcription, speaker diarization, and translation. The model card provides specific instruction templates for each use case (e.g., transcribing only the person closest to the microphone, adding a glossary), making it suitable for meeting minutes, subtitles, customer service call analysis, and audio retrieval.
- It is actionable: Official examples show how to launch an OpenAI-compatible service using vLLM 0.32.0 + NeMo Speech on a single H100 80 GB GPU, or how to call it in-process using the source-code version of Transformers. The license is OpenMDW-1.1, and the card explicitly states that commercial use is permitted. It has received 3057 likes (indicating attention level only).
Usage
- vLLM: Follow the Environment Setup section in the card to install vLLM 0.32.0 and the main branch of NeMo Speech. Set VLLM_PLUGINS=nemo_speechlm, VLLM_MAX_AUDIO_DECODE_DURATION_S=3600, and VLLM_USE_V2_MODEL_RUNNER=0 (required for word-level timestamps). Then launch using the full vllm serve command from the Quick Start guide (including --max-model-len 65536 and MTP speculative decoding configuration). On the client side, use OpenAI chat.completions, place the base64 wav file in audio_url, set the text to "Transcribe the following audio.", set temperature to 0, and disable enable_thinking.
- Transformers: Install via pip install git+https://github.com/huggingface/transformers，AutoModelForMultimodalLM and load using AutoProcessor. Call processor.apply_transcription_request(audio) followed by generate.
- For transcription, translation, and language identification, use temperature 0, disable reasoning, and leave the system prompt empty. For audio question answering, use temperature 0.6, top_p 0.95, and enable reasoning.
Risks
- Mandarin Chinese is only listed as "Supported" in the support matrix (may require fine-tuning) and is not in the "Leading" tier. Audio understanding, reasoning, and spoken instructions have only been evaluated in English.
- This is an offline model; internal chunking does not equate to real-time streaming transcription. The figures cited in the card (transcribing 2238 seconds of audio per second on an RTX PRO 6000, and processing a 30-second recording in 0.13 seconds) come from an optimized version of vLLM within NIM. These optimizations have not yet been included in the open-source version, and NVIDIA estimates the open-source version is approximately twice as slow.
- The repository was created on 10 7 and has been on the leaderboard for some time. Speaker labels only distinguish different speakers in the recording and do not represent real identities.