Sophon-Realtime v1: A full-duplex speech model that understands tone, laughs, and sighs

Homepage for Adventists AI's family of full-duplex speech dialogue models. All three versions share the same 32B model for listening, turn-taking, and response generation, differing only in their vocalization methods: Tag (recommended) has the model generate responses with audio markers, which are then performed by an expressive speech model to produce laughs, sighs, throat-clearing, etc.; Flash uses a lighter, faster speech model, with the official claim of 0.31 seconds for first-packet audio latency per user; E2E is an end-to-end research preview. Officially reported scores are 82.8 on VoiceBench and 81.8% on Big Bench Audio, ranking first among open-source full-duplex models. It supports interruptions and backchanneling, and includes a gateway compatible with the OpenAI GPT-Live protocol, allowing queries and actions to be delegated to backend large models.

Original title: adventists-ai/Sophon-Realtime-v1

Homepage for Adventists AI's family of full-duplex speech dialogue models. All three versions share the same 32B model for listening, turn-taking, and response generation, differing only in their vocalization methods: Tag (recommended) has the model generate responses with audio markers, which are then performed by an expressive speech model to produce laughs, sighs, throat-clearing, etc.; Flash uses a lighter, faster speech model, with the official claim of 0.31 seconds for first-packet audio latency per user; E2E is an end-to-end research preview. Officially reported scores are 82.8 on VoiceBench and 81.8% on Big Bench Audio, ranking first among open-source full-duplex models. It supports interruptions and backchanneling, and includes a gateway compatible with the OpenAI GPT-Live protocol, allowing queries and actions to be delegated to backend large models.

Why it matters Rare in open-source real-time speech dialogue for its ability to detect tone and respond with laughter or sighs; deployable on a single GPU and commercially usable

Upvotes 63 · Likes 65 · Downloads 0

Reasons

  • Detects tonal cues such as hesitation, emphasis, sarcasm, and laughter, and responds with matching tone
  • The Tag and Flash versions officially claim they can be served with a single command using vLLM-Omni on a single 80 GB GPU
  • Code and weights are licensed under Apache-2.0, allowing commercial use; supports fine-tuning personas, tone, and domain knowledge via LoRA

Usage

  • After running pip install sophon-realtime, execute sophon-realtime serve (the first run will download approximately 42 GB of Tag weights)
  • For a quick trial, use the online demo on ModelScope; deployment weights are available in the Sophon-Realtime-32B-Tag and Flash repositories

Risks

  • This repository is the family homepage; deployment requires weights from the Tag or Flash repositories, with the fine-tuning base hosted on ModelScope
  • Most evaluations are self-run by the developers and scored by large models
  • This entry has not been independently tested; refer to the model card for performance details

Open

Hugging Face Trending

Explore the latest AI Radar updates