Duo-1.0-causal-base

Duo consists of two modules: Intent and Realization. User video is segmented via speech recognition and forced alignment and fed into Intent, which determines whether and how to respond on a unified timeline, then guides Realization to generate the assistant's audio and video. This release includes the causal base version of Duo-1.0, the 4-step distilled version, and the FP8 quantized version, as well as the image-quality-improved causal base version of Duo-1.2. Weights are initialized from MiniMax H3. The main branch is an offline pipeline for research purposes, while the vLLM-Omni and SGLang-Omni branches provide real-time streaming deployment.

Duo consists of two modules: Intent and Realization. User video is segmented via speech recognition and forced alignment and fed into Intent, which determines whether and how to respond on a unified timeline, then guides Realization to generate the assistant's audio and video. This release includes the causal base version of Duo-1.0, the 4-step distilled version, and the FP8 quantized version, as well as the image-quality-improved causal base version of Duo-1.2. Weights are initialized from MiniMax H3. The main branch is an offline pipeline for research purposes, while the vLLM-Omni and SGLang-Omni branches provide real-time streaming deployment.

Why it matters Most generative models wait until you finish speaking before producing results; this model uses an autoregressive Intent module to decide when and how to interject, then drives the generation module to produce audio and video, and has open-sourced streaming deployment.

Hugging Face Likes 4 · Hugging Face Trending Score 4 · Hugging Face Downloads 0

License: Conditional Commercial Use · ComfyUI: Not Available · VRAM: Not Specified

Regional License: The MiniMax H3 open-weight license excludes the EU, UK, South Korea, and the US (Excluded Territories in the LICENSE); separate authorization from MiniMax is required in these four regions.

Reasons

  • Includes paper, technical report, browser demo console, and Hugging Face online demo
  • Code is Apache-2.0; ModelScope provides synchronized weights

Usage

  • Clone github.com/inclusionAI/Duo, set up separate Python environments for the model and speech according to the README, run hf download inclusionAI/Duo-1.2-causal-base --local-dir weights, modify configs/inference.json, then run run_demo.py --port 4173, and record a video in the web interface to see how it responds

Risks

  • Preview version; currently supports only basic audio-video dialogue, with visual interaction planned for v2; the default offline configuration requires 2 CUDA GPUs
  • Weights follow the MiniMax H3 community license, which does not apply to the EU, UK, South Korea, and the US

Open

抱脸 world-model 标签

Explore the latest AI Radar updates