An open-source video character/object replacement tool: drag in a video and a character photo to replace the main person in the video with your character, while preserving the camera work, editing, motion, and original audio; it can also change only hairstyles or clothing, remove objects, or swap backgrounds, all running on your own GPU using open weights.
Stars 627 (+265 / 24h)
Reasoning
- A Next.js frontend wrapped around ComfyUI workflows: SAM 3.1 tracks people shot-by-shot, turning the original person into a gray mannequin for MiniMax H3 to animate, redrawing only the masked areas; Qwen3-VL 8B handles scene description and prompt writing (can also be swapped for 4B or Gemma 4 E2B)
- Modes: single-person replacement, advanced multi-person replacement for 1–4 people with role assignment (including a Montage page for seamless person switching), object replacement/removal/addition/background swap, and "from image" mode which uses FLUX to pose first before generating video with relighting; post-processing includes face refinement and softening H3's color tone
- Before submission, it checks against ComfyUI's node and model lists to identify missing components and provides installation links; the Check mask function runs only SAM and masking, allowing frame-by-frame preview of areas to be redrawn
Usage
- Requires an NVIDIA GPU (tested by the author on RTX 4060 Ti 16 GB + 64 GB RAM), ComfyUI 0.41.2 or higher, and Node.js 22.6 or higher; follow the README to install 12 custom node packages and files such as the H3 pruned INT8 model, DMAD acceleration LoRA, and SAM 3.1
- On Windows, double-click start_openjutsu.bat; on other systems, first start ComfyUI (on Windows, set AIOHTTP_NOSENDFILE=1), install comfy-api-proxy==0.1.6 and patch it using the repository's patch_proxy.py, then run npm ci, npm run build, and npm start, and open 127.0.0.1:3000
- Defaults are 736p and 6 steps; the author states that on a 4060 Ti, a 480p 6-second clip takes about 6–13 minutes, and 1080p is about 2.5 times slower than 736p
Risks
- It can replace real people with anyone from a photo; obtain consent before using someone else's likeness; currently, people in mirrors or reflections cannot be replaced, and H3 may copy the original person's appearance from reflections
- Face refinement relies on InsightFace buffalo_l, which is licensed for non-commercial research only, and Bingsu/adetailer's YOLO weights are AGPL-3.0; the author has not yet tested the deployment process to Comfy cloud platforms; the code is MIT licensed, the repository was created on 10/26, stars are growing rapidly, and it is a normal tool repository with a reviewed README