Original title: fishaudio/drama-3
Fish Audio's next-generation text-to-speech model designed for multi-character scenarios and long-form content. The text backbone is an MoE (with 350 billion total parameters, activating 30 billion per token, mixing full attention and linear attention), while the audio decoder performs causal decoding across 10 codebooks; using tags from <|speaker:0|> to <|speaker:15|>, a single text segment can be divided into up to 16 characters. It supports voice cloning, inserting event tags such as [sighs] and [laughs] within the text, and using the text from a previous recording as context to regenerate with only one word changed. Officially, it claims that on a single H200, the first audio packet for a single stream takes approximately 89 milliseconds, and even with 16 concurrent streams, the generation speed remains approximately 2.7 times the playback speed.
Why It Matters Suitable for audiobooks, radio dramas, and multi-person dialogue dubbing; changing one word does not require re-recording the entire segment, and the first audio packet for a single stream takes less than 100 milliseconds
Rising Score 59 · Likes 1249 · Downloads 46208
Reasons
- Switch between multiple speakers within a single text segment; long texts are automatically segmented and can be streamed by segment, making it suitable for radio dramas, audiobooks, and multi-person dialogues
- "Continuation-style editing": Place previous versions in previous_takes and change only the words that need modification, allowing for multiple consecutive edits
- Day-one support for SGLang-Omni and vLLM-Omni, accompanied by a technical report and performance tables under different concurrency levels
Usage
- SGLang-Omni: After installing fishaudio additional dependencies, run sgl-omni serve --model-path fishaudio/drama-3; or run vllm serve fishaudio/drama-3 --omni
- In requests, provide reference audio and corresponding text in references to clone the voice timbre; write tags such as [sighs] and [laughs] directly in the text to control performance
Risks
- Fish Audio Community License: Free for research and non-commercial use; commercial use is permitted if annual revenue is below 50 ten thousand USD; providing external TTS / voice cloning hosting services requires a separate license
- Performance data was measured by the official team on H200
- Authorization must be obtained before using someone else's voice
- This entry has not been run; refer to the model card for actual performance