Original title: Efficient-Large-Model/Open-H3-Regenerate-2K-IC
Video regeneration tool released by Efficient-Large-Model (code in the NVlabs/Sana repository): Use the 768P draft generated by MiniMax-H3 as a reference video in the context of the Ref2VA transformer, then use a Context (IC) LoRA to generate the 1080P, 2K (2560×1440) or 4K (3840×2160) version of the same segment, preserving faces, lip-sync, text, and layout from the draft. By default, it first upscales the draft using a latent space amplifier, then runs the last 3 steps of the HyperFlow 8-step grid; you can also choose 4 steps, or run the full 8 steps from noise, all with CFG 1. The repository includes inference code, download scripts, and a script to merge LoRAs, facilitating environments that can only load a single LoRA.
Why it matters You can quickly test shots at low resolution first, then produce the high-definition final version once satisfied, without changing the visual layout or lip-sync
Upvotes 60 · Likes 60 · Downloads 11312
Reasons
- Suitable for the workflow of "trial and error at low resolution, then final output at high resolution"; the author states that you can simply reuse the original prompt used to generate the draft, or leave it empty
- The repository packages the HyperFlow 8-step LoRA, LMS Sharp sharpening LoRA, latent space amplifier, and fast VAE decoder, all fetched at once via the download script
Usage
- After running pip install -r infer/requirements.txt, run python infer/download.py --out weights, then use torchrun to run infer.py, specifying the draft video with --input and filling in the original prompt with --caption
- Choose 2k (default), 1080p, or 4k for --resolution; choose up3 (default), up4, or hf8 for --sampler; pass a jsonl file to --jobs for batch processing
Risks
- The example command uses 8 GPUs; the transformer can be sharded across 1, 2, 4, 7, or 8 cards, but the author has not specified the single-GPU VRAM requirement
- The MiniMax H3 community license does not cover the EU, UK, South Korea, and the US; this is not an official MiniMax product, and third-party components like LMS Sharp have separate terms
- This entry has not been run; refer to the project page for actual results