This is an Apple MLX weight conversion of Meituan's LongCat-Video-Avatar 1.6 by community author fedeinfe: input a portrait (or a few frames of real footage) and an audio clip to output a talking video of any length. 1.6 retains the architecture of 1.5, but replaces it with a retrained 136 billion-parameter DiT and two distilled LoRAs. It follows Meituan's recommended 8-step sampling, using DMD for the first 4 steps and OPSD for the last 4 steps; long audio is stitched into a continuous shot in segments of 93 frames (25 fps). The author measured that on an M5 Max with 64 GB, generating a 3.7-second video at 480p takes about 8 minutes, with peak memory usage around 47 GB.
Why it matters It runs locally on Apple Silicon Macs, automatically stitches long audio into a continuous shot, and includes scripts for training custom lip-sync LoRAs; however, it is memory-intensive and slow to render.
Hugs: 3 · Upvotes: 3 · Downloads: 0
License: Commercial use allowed · ComfyUI: Not available · VRAM: Not specified
Reasons
- MIT license, consistent with Meituan's original; the DiT weights are a bit-for-bit direct conversion of Meituan's bf16 weights
- The model card provides speed, memory, and lip-sync scores for two resolutions: 480p and 1120×832
Usage
- git clone -b feat/avatar-1.6-continuation-lora https://github.com/fedeinfe/longcat-avatar-mlx,建 Set up a Python 3.12 virtual environment and run pip install -e ".[parity]" librosa pyloudnorm pillow ftfy, download the weights to the weights directory, then run scripts/generate.py, passing --image for the portrait, --audio for the voice, --prompt for the description, and --out for the output file
Risks
- Supports only Apple Silicon Macs, requires 64 GB of unified memory, and the weights are 51.5 GB; a segment at 1120×832 takes about 45 minutes
- The inference code is in the author's branch, and the merge request to the original repository has not been merged; feeding audio at 30 fps will cause lip-sync issues, so you must generate at 25 fps and then interpolate frames
- Obtain consent from the individuals before generating videos using their portraits and voices