Audio8-TTS-Realistic-Air: Small-parameter multilingual speech synthesis

A speech synthesis model from Edge0, continued training based on Audio8-TTS-Preview-0.6b. The main model has 6.01 billion parameters (excluding the codec) and adopts a DualAR architecture inspired by Fish Audio S2 Pro: a slow autoregressive transformer predicts one semantic token per frame, while a fast autoregressive transformer predicts the 10 codebooks for that frame. Supported languages have increased from 11 in the preview version to 23, adding Arabic, Finnish, Hindi, Indonesian, Malay, Norwegian, Portuguese, Russian, Swedish, Thai, Turkish, and Vietnamese. It supports zero-shot voice cloning with reference audio and transcripts, or direct synthesis without references; the repository includes a 44.1kHz codec.

Original title: Edge0/Audio8-TTS-Realistic-Air

A speech synthesis model from Edge0, continued training based on Audio8-TTS-Preview-0.6b. The main model has 6.01 billion parameters (excluding the codec) and adopts a DualAR architecture inspired by Fish Audio S2 Pro: a slow autoregressive transformer predicts one semantic token per frame, while a fast autoregressive transformer predicts the 10 codebooks for that frame. Supported languages have increased from 11 in the preview version to 23, adding Arabic, Finnish, Hindi, Indonesian, Malay, Norwegian, Portuguese, Russian, Swedish, Thai, Turkish, and Vietnamese. It supports zero-shot voice cloning with reference audio and transcripts, or direct synthesis without references; the repository includes a 44.1kHz codec.

Why it matters The main model has only six hundred million parameters, is Apache-licensed, includes SGLang Omni service adaptation and an OpenAI-compatible speech interface, making it suitable for local or batch dubbing.

Rising score 52.5 · Likes 514 · Downloads 18680

Reasons

  • Officially, Seed-TTS has an average error rate of 1.25% for Chinese and English (preview version 1.23%), with speaker similarity rising from 0.682 to 0.704; the average error rate across nine languages on CV3 is 4.86%.
  • The official report lists error rates by language individually, rather than providing only averages: new languages such as Thai, Finnish, and Vietnamese have higher error rates on CV3.
  • There is a ZeroGPU online trial Space; the SGLang Omni adaptation supports paged attention, dynamic batching, and the /v1/audio/speech interface.

Usage

  • After installing torch, transformers (version 4.57 or higher, but below 5), and soundfile via pip, use AutoModel / AutoProcessor.from_pretrained("Edge0/Audio8-TTS-Realistic-Air", trust_remote_code=True); when cloning voices, pass reference_audio and an exactly matching reference_text.
  • For batch services, follow the sglang_omni instructions in the GitHub repository Edge0-AI/Audio8_TTS, using the audio8_tts_0_6b.yaml configuration and replacing the model path with this repository.

Risks

  • Quality varies significantly across languages, with notably higher error rates for Cantonese and some newly added languages; the official team warns of potential mispronunciations, repetitions, omissions, or early stopping, so manual listening is required before finalizing output.
  • The official documentation states that these metrics only measure intelligibility and voice similarity, not directly "realism"; comparisons with other models come from different evaluation pipelines and are for reference only.
  • Running custom code requires trust_remote_code; voice cloning requires permission; this entry has not been run.

Open

Hugging Face Trending

Explore the latest AI Radar updates