Original title: Viggle/Viggle-Remake
Viggle trained two rank 256 LoRAs on MiniMax-H3: swap handles character/role replacement (multiple at once), while edit handles all other changes—adding, removing, or replacing objects, and changing backgrounds, weather, lighting, and style. The inputs are the original video, 1–4 hand-drawn keyframes (referred to as Picture N in the prompt, which can be pinned to a specific frame of the original video using --pin), and a six-part prompt for MiniMax-H3. The output preserves the original video's motion, camera work, pacing, and audio. The repository also includes an MCP service: hand the video and a single requirement to coding agents like Claude Code or codex, and they will automatically cut shots, draw keyframes, write prompts, render window-by-window on your GPU, check results, and stitch everything back together with the original audio. This is the Viggle-Animate v2 previewed on the Viggle-Animate card. The repository was created on 10 22, and was not included in previous issues.
Why it matters Handing shot-by-shot manual video editing to agents: cutting shots, drawing keyframes, rendering, self-checking, and stitching back the original audio are all automated
Rising score 95 · Likes 97 · Downloads 5215
Reasons
- Addresses the need to "change actors or scenes in existing footage without reshooting": the author's examples remake a single shot from La La Land into 22 versions (changing people, costumes, weather, adding/removing objects, changing style), along with a full remake of a 116-second MV with 69 edit points and a 101-second clip from The Matrix 2, all preserving the original soundtrack.
- The agent-driven full-video workflow is the most distinctive feature of this repository: it renders a single window of 90–345 frames at a time (24 fps, approximately 12 seconds); for longer videos, the MCP attempts to cut windows at edit points. The author states that having codex complete the White House Funk example takes about 1.5 hours on two GPUs, with keyframes drawn by the agent itself each time, resulting in different details.
- There is a path for smaller GPUs: the ComfyUI node package only adds four things that the built-in nodes cannot do (reference image presentation, pinning keyframes by actual frame number, clean preservation of original audio, and generating starting noise based on seed); everything else uses built-in nodes. The author also released int8 (21.0 GB) and GGUF (Q8_0 22.3 GB, Q6_K 17.6 GB, Q4_K_M 14.3 GB) single files where the LoRA is merged into the base model under fp32 and then quantized; low-rank LoRAs with rank 128 / 64 are only 1.34 / 0.67 GB, and the author claims the output is nearly identical.
Usage
- Your own GPU: Download the MiniMaxAI/MiniMax-H3 base model (a file of approximately 144 GB) and the lora/, inference/, examples/, and requirements.txt (5.5 GB) from this repository. Run pip install -r requirements.txt and ensure ffmpeg is installed. Run python Viggle-Remake/inference/sample.py --mode swap (or edit) --video source.mp4 --prompt prompt.txt --ref ref1.png --pin 12 --frames 124 --short-edge 480 --seed 0 --base ./MiniMax-H3 --lora-repo ./Viggle-Remake --out out.mp4; --frames should be 17n+5, and --short-edge 736 outputs 1280×736. Do not git clone the entire repository (using git-lfs will download approximately 210 GB). You can run examples/demo.sh first to verify that the LoRA is loaded with the built-in examples.
- ComfyUI (tested by the author on 0.42.0): Use the fl2va single file from Comfy-Org/MiniMax-H3 as the base model (not ref2va). Copy comfyui/ComfyUI-Viggle-Remake into custom_nodes, place the LoRA in models/loras, and load it using LoraLoaderBypassModelOnly—the author claims that the standard LoraLoaderModelOnly loses a significant portion of LoRA updates under bf16, resulting in softer images. Drag comfyui/workflows/remake_swap_ui.json or remake_edit_ui.json to run the examples; convert your own assets to 24 fps first. When using int8 / GGUF merged files, they replace both the base model and the LoRA, so do not stack another LoRA on top.
- Agents: Use hf download Viggle/Viggle-Remake --include "mcp/*" to fetch only the MCP (without weights). In the mcp directory, run pip install -e . (on GPU machines, also install '.[worker]'). Start one viggle-remake-worker per GPU, then connect the service to Claude Code or codex, providing it with a video and a single requirement. The author details how to draw keyframes, when to pin frames, and how to write six-part prompts in the repository's PROMPTING.md. If you do not want to deploy it yourself, you can use the Viggle app.
Risks
- High barrier to entry: sample.py requires a single GPU with 80 GB or more (bf16); the author measured that a single ComfyUI render takes about 95 seconds and uses about 130 GB of system memory on B200. The base model and its text encoder are mandatory.
- Known issues listed by the author: non-humanoid characters in long shots fail, there are minor detail artifacts, and multi-shot clips drift temporally if keyframes are not pinned; audio is simply carried over from the original, so silent inputs produce silent outputs.
- The character replacement capability includes no built-in identity or consent verification; identity is entirely determined by the keyframes you draw. The author requires that it be used only on material you have the right to modify, not on people who have not consented, and that generated content be labeled as AI. Copyright for the movie and MV clips in the examples belongs to the original owners, and neither license grants any rights to these materials or likenesses.
- The weights are under the MiniMax H3 Community License (commercial product interfaces must prominently display MiniMax H3, and annual revenue exceeding 2000 ten thousand USD requires prior written authorization from MiniMax); the code is Apache-2.0. When building products based on this, the author requests attribution with "Powered by Viggle Remake".