Open-H3-Regenerate: Regenerates H3 draft videos to 4K

A video regeneration tool released by Efficient-Large-Model (code in the NVlabs/Sana repository): It uses the 768P draft generated by MiniMax-H3 as a reference video within the context of the Ref2VA transformer, then employs an In-Context (IC) LoRA to generate a 1080P, 2K (2560×1440) or 4K (3840×2160) version of the same segment, preserving faces, lip-sync, text, and layout from the draft. By default, it first upscales the draft using a latent space amplifier, then runs the last 3 steps of the HyperFlow 8-step grid; alternatively, you can choose 4 steps or run the full 8 steps from noise, all with CFG 1. The repository includes inference code, download scripts, and a script to merge LoRAs, facilitating environments that can only load a single LoRA.

Original title: Efficient-Large-Model/Open-H3-Regenerate-2K-IC

A video regeneration tool released by Efficient-Large-Model (code in the NVlabs/Sana repository): It uses the 768P draft generated by MiniMax-H3 as a reference video within the context of the Ref2VA transformer, then employs an In-Context (IC) LoRA to generate a 1080P, 2K (2560×1440) or 4K (3840×2160) version of the same segment, preserving faces, lip-sync, text, and layout from the draft. By default, it first upscales the draft using a latent space amplifier, then runs the last 3 steps of the HyperFlow 8-step grid; alternatively, you can choose 4 steps or run the full 8 steps from noise, all with CFG 1. The repository includes inference code, download scripts, and a script to merge LoRAs, facilitating environments that can only load a single LoRA.

Why it matters You can quickly test shots at low resolution first, then produce the high-definition final version once satisfied, with no changes to layout or lip-sync.

Upvotes 60 · Likes 60 · Downloads 11312

Reasons

  • Suitable for a workflow of "low-res trial and error first, high-res output later"; the author notes that you can simply reuse the original prompt used to generate the draft, or leave it empty.
  • The repository packages the HyperFlow 8-step LoRA, LMS Sharp sharpening LoRA, latent space amplifier, and fast VAE decoder, all retrievable at once via the download script.

Usage

  • After running pip install -r infer/requirements.txt, execute python infer/download.py --out weights, then run infer.py using torchrun. Use --input to specify the draft video and --caption to fill in the original prompt.
  • Use --resolution to select 2k (default), 1080p, or 4k; use --sampler to select up3 (default), up4, or hf8; pass a jsonl file via --jobs for batch processing.

Risks

  • The example command uses 8 GPUs. The transformer can be sharded across 1, 2, 4, 7, or 8 cards, but the author has not specified the single-GPU VRAM requirement.
  • The MiniMax H3 community license does not cover the EU, UK, South Korea, and the US; this is not an official MiniMax product, and third-party components like LMS Sharp have separate terms.
  • This entry has not been tested; refer to the project page for actual results.

Open

Hugging Face Trending

Explore the latest AI Radar updates