Wave Forcing: Approaching the "Speed of Light" in Streaming Long-Video Generation (Towards Speed-of-Light Streaming Video Generation)

A real-time long-video generation solution from Deng Zhijie's group at Shanghai Jiao Tong University, co-designed across generation, training, and execution. Existing chunked autoregressive methods require fully denoising one chunk before refreshing the history cache, necessitating two forward passes even for single-step denoising. Wave Forcing employs chunked causal attention, allowing subsequent chunks to read "cleaner" context while preceding chunks are not yet fully denoised. It then distributes staggered denoising stages and cache refreshes across full model replicas on multiple GPUs. Once the pipeline is filled, each round of concurrent forward passes outputs one chunk (referred to by the authors as 1F1C). Training consists of three stages: mixed-noise denoising, motion supervision, and causal adaptation using self-generated history. At runtime, WaveRT uses Wave Parallelism to proactively push K/V tensors for each layer in a unidirectional manner, hiding communication behind attention computation, and pipelines VAE decoding.

A real-time long-video generation solution from Deng Zhijie's group at Shanghai Jiao Tong University, co-designed across generation, training, and execution. Existing chunked autoregressive methods require fully denoising one chunk before refreshing the history cache, necessitating two forward passes even for single-step denoising. Wave Forcing employs chunked causal attention, allowing subsequent chunks to read "cleaner" context while preceding chunks are not yet fully denoised. It then distributes staggered denoising stages and cache refreshes across full model replicas on multiple GPUs. Once the pipeline is filled, each round of concurrent forward passes outputs one chunk (referred to by the authors as 1F1C). Training consists of three stages: mixed-noise denoising, motion supervision, and causal adaptation using self-generated history. At runtime, WaveRT uses Wave Parallelism to proactively push K/V tensors for each layer in a unidirectional manner, hiding communication behind attention computation, and pipelines VAE decoding.

Likes 106 · Code Stars 96

Rationale

  • The authors claim that on Wan2.1-T2V-1.3B, the 4-step version achieves DiT 147.6 FPS and end-to-end 134.8 FPS on 8 H200 GPUs, representing 7.07 times the end-to-end throughput of a single GPU. The README states that decoding 1593 frames (approximately 100 seconds at 16 FPS) takes about 11.8 seconds.
  • Regarding quality, the authors claim the 5-step version is overall comparable to Rolling Forcing, with higher Dynamic Degree (motion amplitude) for both short and long videos. In pure DiT comparisons, under the same backend, step count, and GPU count, Wave Parallelism achieves the highest throughput, which is 2.53 times that of the faster of Ulysses or Ring context parallelism (across six model sizes from 1.3B to 14B).
  • It is runnable: Inference code and the WaveRT runtime are Apache-2.0 licensed. The Hugging Face repository SJTU-DENG-Lab/WaveForcing-1.3B contains 4-step and 5-step BF16 checkpoints (with mirrors on ModelScope). Reference training code has also been released. It ranked second in likes on Hugging Face on the day of release (indicating attention only).

Usage

  • Download: hf download Wan-AI/Wan2.1-T2V-1.3B-Diffusers --local-dir ckpts/Wan2.1-T2V-1.3B-Diffusers, then hf download SJTU-DENG-Lab/WaveForcing-1.3B --local-dir ckpts/WaveForcing-1.3B (the text encoder and VAE are read from the Wan2.1 base model).
  • 5 steps (5+2 layout): python -m wave\_rt --model-path ckpts/Wan2.1-T2V-1.3B-Diffusers --gen-ckpt ckpts/WaveForcing-1.3B/5step/model.safetensors --rf-step 5 --wp-size 6 --denoising-step-list 1000,800,600,400,200 --num-frames 24 --height 480 --width 832 --vae-stages 2 --cuda-visible-devices 0,1,2,3,4,5,6,7 --prompt "…". Output is located in outputs/quickstart/\<run-tag\>/video.mp4.
  • 4 steps (4+3 layout, i.e., the headline figures): Point --gen-ckpt to 4step/model.safetensors, and change to --rf-step 4 --wp-size 5 --denoising-step-list 1000,750,500,250 --vae-stages 3.
  • --num-frames specifies the number of latent frames and must be divisible by 3. Setting it to 399 corresponds to the long-video benchmark (decoding 1593 frames).

Risks

  • High barrier to entry: The README's quick start is written for a 8-GPU pipeline. The environment has only been verified on Python 3.12, PyTorch 2.11.0+cu128, the SGLang diffusion runtime, and Hopper GPUs. There is no documentation on how to run it on a single GPU or consumer-grade GPUs.
  • Currently, only 1.3B checkpoints are available at a resolution of 832×480. The 14B row in the README table uses randomly initialized weights, testing only system scalability and not representing generation quality.
  • The reference training code follows the academic-use license of RollingForcing, which differs from the Apache-2.0 license of the runtime. FPS and quality comparisons are author-reported; since the generated videos have not been viewed, no assessment of visual quality is provided.

Editor's Note: Suitable for teams with multiple H100 / H200 GPUs who want to perform real-time long-video streaming or research chunked autoregressive video parallelism. Single-GPU users should review the full configuration matrix on the project page before deciding.

Open · Code · Hugging Face Paper Page

Hugging Face Daily Papers

Explore the latest AI Radar updates