Fish Audio's multi-speaker speech synthesis model, previously indexed on 10 23: a single text segment can be assigned to up to 16 speakers, supports voice cloning, allows tone control via tags like [sighs] and natural language, and enables re-recording with modified text based on previous audio versions. This re-indexing is due to the main repository's weights being completely updated on 10 29.
Rising score 111 · Likes 1127 · Downloads 31923
Reasons
- New content: On 10 29, the main repository uploaded a complete set of new weights, with the number of shards changing from 8 to 15. The upload description stated these were weights from "RL iteration 100", with the FP8 format and file layout remaining unchanged; this description was subsequently deleted. Versions downloaded before 10 29 need to be re-fetched to obtain the new version.
- License correction: On 10 23, this column noted that "LICENSE.md could not be opened, making commercial terms unverifiable." In fact, the LICENSE file in the repository is readable; it is the Fish Audio Community License 1.0: free for research and non-commercial use; free for commercial use by individuals or companies with annual revenue below 50 ten thousand USD; providing external TTS APIs or hosting services requires a separate agreement regardless of scale.
- The technical report (arXiv 2610.18010) states that the reinforcement learning stage uses only a learned human preference model for reward; the inference engine decodes 271 frames per second per stream on a single H200 GPU, while Fish Audio S2 decodes 177 frames (all figures as claimed by the official source).
Usage
- SGLang-Omni: git clone sgl-project/sglang-omni, uv pip install --prerelease=allow -e ".[fishaudio]", then sgl-omni serve --model-path fishaudio/drama-3 --port 8000. You can also use vLLM-Omni: vllm serve fishaudio/drama-3 --omni --port 8091.
- Call /v1/tts: pass reference audio and transcripts in
referencesto clone voices; use <|speaker:0|> to <|speaker:15|> in the text to assign roles, and insert tags like [sighs] or [laughs] at positions where sound should occur. For re-recording with modified text, place the old text inprevious_takes, allowing multiple consecutive modifications. - For high concurrency, the official recommendation is to place the vocoder on a second GPU: with 256 concurrent streams, throughput increases by approximately 72%, and the median first-packet latency decreases by 37%.
Risks
- The weight update lacks release notes or comparisons between old and new versions; the official source has not explained what changed or what improvements were made. This column has not listened to samples from either the old or new versions.
- The model card still links to a non-existent LICENSE.md at the beginning; terms must be checked in the LICENSE file within the repository. Hosting/API services and commercial use with annual revenue exceeding 50 ten thousand USD require a separate agreement.
- The main weights are approximately 39.6 GB (expert layers are FP8); official benchmarks are all on H200 GPUs, with no information provided on whether consumer-grade GPUs can run it; ComfyUI support is currently unavailable.