Original title: Edge0/Audio8-TTS-Realistic-Air
A speech synthesis model from Edge0, continued training on Audio8-TTS-Preview-0.6b. The main model has 0.6B parameters (601,159,424 parameters, excluding the separate 44.1 kHz codec). The authors claim more natural speech and more expressive prosody. It supports 23 languages, adding Arabic, Finnish, Hindi, Indonesian, Malay, Norwegian, Portuguese, Russian, Swedish, Thai, Turkish, and Vietnamese—12 new languages compared to Preview—including Chinese, Cantonese, English, Japanese, and Korean. You can clone a voice in zero-shot mode by providing a reference recording and its verbatim transcript, or synthesize directly without a reference. The architecture follows Fish Audio S2 Pro's DualAR: the slow AR predicts one semantic token per frame, while the fast AR predicts 10 codebooks for that frame. Added to the repository on 10/14; not included in previous issues.
Rising score 252 · Likes 318 · Downloads 11349
Reasons
- Small model, multilingual: The authors report an average error rate of 1.25% for Seed-TTS in Chinese and English (Preview: 1.23%) and a speaker similarity of 0.704 (Preview: 0.682). On CV3, the error rate for eight shared languages dropped from 4.80% to 4.56%. The authors also note that these metrics measure intelligibility and similarity, not directly "realism."
- Complete deployment path: Direct loading via Transformers (the codec is bundled, no need to find it separately). Additionally, the SGLang Omni plugin provides paged attention, dynamic batching, and an OpenAI-compatible /v1/audio/speech interface. The GitHub repository also includes command-line tools, batch processing scripts, and supervised fine-tuning scripts.
- Apache-2.0, with a ZeroGPU online trial Space; Hugging Face likes 318 (indicating attention only).
Usage
- pip install "torch>=2.5.0" "torchaudio>=2.5.0" "transformers>=4.57.0,<5" soundfile safetensors; AutoProcessor / AutoModel.from_pretrained('Edge0/Audio8-TTS-Realistic-Air', trust_remote_code=True). For cloning, provide reference_audio and reference_text that matches the recording verbatim. Use generate(max_new_tokens=1024, temperature=0.8, top_p=0.95, top_k=50, do_sample=True, return_dict_in_generate=True), then use model.decode_audio(output.codes) to write a wav file.
- When no reference voice is needed, pass only text to the processor. Reference recordings should be clean; split long texts into segments before synthesis (reference audio, transcript, and target text share context). For serving, use the sglang_omni configuration audio8_tts_0_6b.yaml from GitHub Edge0-AI/Audio8_TTS, setting the model path to this repository.
Risks
- Significant variation across languages: In the authors' CV3 table, Chinese CER is 1.814%, English WER is 2.907%, but Thai is 48.574%, Finnish 32.279%, and Vietnamese 28.111%. The CER for difficult Chinese cases is slightly higher than Preview (11.510% → 12.797%).
- SGLang serving has only been verified to run on H20 (non-streaming, with torch.compile and CUDA Graph disabled); throughput and streaming have not been tested. The authors warn of potential mispronunciations, repetitions, omissions, or early stopping; listen to each output individually before finalizing.
- Obtain permission before cloning someone else's voice, and label synthesized audio appropriately for the context. No mention of watermarking in the card.