Original title: adventists-ai/Sophon-Realtime-v1
The open-source full-duplex speech model family from Adventists AI (AI Adventists), 11-10 is now included. All three versions share the same 32B “brain” responsible for listening, turn-taking judgment, and generating responses; the only difference lies in how responses are converted into audio: Tag uses a more expressive speech model, Flash is faster with higher concurrency per GPU, and E2E uses a speaker head with approximately 3.5 hundred million parameters to directly generate continuous audio vectors. Progress since 11-10: On 11-11, the E2E version (1.1.3) transitioned from preview to stable release, reducing hardware requirements from two GPUs to one 80GB GPU; on 11-12 (1.1.4), E2E can also be deployed as a vLLM-Omni model, using the same commands and interfaces as Tag and Flash; on 11-13 (1.2.0), long responses are no longer truncated at approximately 2 minutes; on 11-14 (1.3.0), E2E supports zero-shot voice cloning, requiring only a 5–12 second recording, with unchanged weights.
Why it matters Continuous updates since the last inclusion: The E2E version can be deployed using the same interface as Tag and Flash, long responses are fully generated, and a free trial hosted API is available
Score increase 158 · Likes 210 · Downloads 0
Reasons
- The official team states that the shared brain achieves VoiceBench 82.8 and Big Bench Audio 81.8%, ranking first among open-source full-duplex models; E2E has a first-packet latency of approximately 0.25 seconds per stream, with a recommended concurrency of 16 streams per GPU
- The official team provides non-verbal vocalization evaluations for E2E: NVV-SuperBench Chinese 0.85, English 0.75 (combined scores for laughter, coughing, sighing, and breathing)
- A hosted API has been added to the homepage, offering 3 hours of free voice conversation upon registration, compatible with the OpenAI Realtime protocol, allowing testing without a GPU
Usage
- For E2E using vLLM-Omni: git clone the https://github.com/adventists-ai/sophon-realtime,进 e2e/ directory and run bash install.sh (approximately 41GB), then run GPU=0 PORT=8000 bash start.sh; the interfaces are /v1/realtime and /v1/chat/completions
- Choose Tag for expressiveness priority, or Flash for higher concurrency per GPU (after pip install "sophon-realtime[trt]", run sophon-realtime serve); Live mode is compatible with the OpenAI GPT-Live protocol, allowing queries and actions to be delegated to larger backend models
Risks
- Requires an H100 class 80GB GPU (E2E actually occupies approximately 70GB); the official team states that on RTX PRO 6000, E2E can only support approximately 4–8 concurrent streams
- The E2E version supports only Chinese and English; evaluations are self-reported by the official team, and some comparison scores are taken from third-party leaderboards
- Zero-shot voice cloning requires code 1.3.0 or higher and is only available via the vLLM-Omni method; this feature has not been tested in this entry