NVIDIA released the open-weight speech model Nemotron 3.5 Transcribe on 10 28: a single model that simultaneously performs transcription, speaker diarization, translation, and audio question answering, with the output determined by a single natural language instruction; it can transcribe 40 languages, of which 31 can be translated to and from English and 13 support speaker diarization, handling recordings up to 1 hours long with up to 8 speakers. It also released the Granary v2 data cleaning pipeline and data manifest, along with a NeMo Speech fine-tuning tutorial.
Confidence: High (100) · Event Time 2026-10-28
Rationale
- With the same meeting recording, changing the prompt can yield a verbatim transcript with speaker labels, an English translation, or an answer to "What decisions were made in this meeting?"; you can also provide a domain-specific vocabulary (e.g., drug names) in the prompt, which the model uses only if those terms actually appear in the recording. A workflow that previously required chaining together transcription, speaker diarization, and translation models can now be completed with a single model.
- The architecture consists of a speech encoder + a hybrid Mamba–Transformer mixture-of-experts language model, with approximately 30B total parameters and approximately 4B active parameters per token (the model card states approximately 4.3B, including the audio encoder), available in BF16, FP8, and NVFP4 precisions. Official claims: #1 among open-weight models on the Hugging Face Open ASR English leaderboard (average word error rate 5.33%); tied for #1 among open-weight models on the Artificial Analysis Speech-to-Text leaderboard (2.8%), where competitors have 24B active parameters; lowest error rate (18.26%) on David AI's speaker diarization leaderboard for 21 languages.
- The official documentation provides a complete domain adaptation example: on a medical speech dataset with Indian doctor accents, crafting a prompt reduced the word error rate from 30.8% to 23.8%, and further fine-tuning reduced it to 11.8%; on speakers not involved in tuning, it dropped from 22.44% to 12.17% (a relative reduction of 45.8%), with the entire adaptation process taking approximately 14 GPU hours on a single GB300.
- The Granary v2 public manifest covers 463,062 hours, 48 languages, and 16 source corpora (v1 covers 25 languages), providing three transcription formats (verbatim, normalized, etc.) for each recording, and the pipeline can be used to process your own recordings.
Usage
- The model is available on Hugging Face at nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B (synchronized on ModelScope), with an online demo; it can be self-hosted using vLLM or SGLang-Omni, or accessed via managed endpoints from Baseten, DeepInfra, and Simplismart. Quantized versions and local deployment via NeMo-Speech.cpp are officially stated to be "coming soon."
- The data is available on Hugging Face at nvidia/Granary-v2; for fine-tuning, refer to the NeMo Speech tutorial and the medical ASR case study, along with an agent skill that allows coding agents to run the full fine-tuning workflow. The model card provides recommended prompts for each task and a language support matrix.
Risks
- It only processes pre-recorded audio, not real-time streaming recognition; audio question answering, reasoning, and voice commands have only been evaluated in English.
- The license is OpenMDW-1.1, and the model card states it is available for commercial use; the model card also notes that performance on pure text tasks is lower than that of its base language model.
- The Decoder reports Granary v2 as approximately 84.4 ten-thousand hours, which is inconsistent with the 463,062 hours stated in the NVIDIA blog; defer to the official source. SiliconANGLE's description of "capable of processing 100 hours of audio" does not match the official original text.
- Leaderboard results are compiled by the official team, and the actual model outputs have not been independently verified.
Open · Nemotron 3.5 Transcribe Model Card · NVIDIA Blog (RTX Spark Local AI Summary) · TechCrunch · The Decoder · SiliconANGLE