Original title: Efficient-Large-Model/Open-H3-Regenerate-2K-IC
A video regeneration tool released by Efficient-Large-Model (code in the NVlabs/Sana repository): It uses the 768P draft generated by MiniMax-H3 as a reference video within the context of the Ref2VA transformer, then employs an In-Context (IC) LoRA to generate a 1080P, 2K (2560×1440) or 4K (3840×2160) version of the same segment, preserving faces, lip-sync, text, and layout from the draft. By default, it first upscales the draft using a latent space amplifier, then runs the last 3 steps of the HyperFlow 8-step grid; alternatively, you can choose 4 steps or run the full 8 steps from noise, all with CFG 1. The repository includes inference code, download scripts, and a script to merge LoRAs, facilitating environments that can only load a single LoRA.
Why it matters You can quickly test shots at low resolution first, then produce the high-definition final version once satisfied, with no changes to layout or lip-sync.
Upvotes 60 · Likes 60 · Downloads 11312
Reasons
- Suitable for a workflow of "low-res trial and error first, high-res output later"; the author notes that you can simply reuse the original prompt used to generate the draft, or leave it empty.
- The repository packages the HyperFlow 8-step LoRA, LMS Sharp sharpening LoRA, latent space amplifier, and fast VAE decoder, all retrievable at once via the download script.
Usage
- After running pip install -r infer/requirements.txt, execute python infer/download.py --out weights, then run infer.py using torchrun. Use --input to specify the draft video and --caption to fill in the original prompt.
- Use --resolution to select 2k (default), 1080p, or 4k; use --sampler to select up3 (default), up4, or hf8; pass a jsonl file via --jobs for batch processing.
Risks
- The example command uses 8 GPUs. The transformer can be sharded across 1, 2, 4, 7, or 8 cards, but the author has not specified the single-GPU VRAM requirement.
- The MiniMax H3 community license does not cover the EU, UK, South Korea, and the US; this is not an official MiniMax product, and third-party components like LMS Sharp have separate terms.
- This entry has not been tested; refer to the project page for actual results.