LongCat-Video-Avatar-1.6-MLX-bf16

This is an Apple MLX weight conversion of Meituan's LongCat-Video-Avatar 1.6 by community author fedeinfe: input a portrait (or a few frames of real footage) and an audio clip to output a talking video of any length. 1.6 retains the architecture of 1.5, but replaces it with a retrained 136 billion-parameter DiT and two distilled LoRAs. It follows Meituan's recommended 8-step sampling, using DMD for the first 4 steps and OPSD for the last 4 steps; long audio is stitched into a continuous shot in segments of 93 frames (25 fps). The author measured that on an M5 Max with 64 GB, generating a 3.7-second video at 480p takes about 8 minutes, with peak memory usage around 47 GB.

This is an Apple MLX weight conversion of Meituan's LongCat-Video-Avatar 1.6 by community author fedeinfe: input a portrait (or a few frames of real footage) and an audio clip to output a talking video of any length. 1.6 retains the architecture of 1.5, but replaces it with a retrained 136 billion-parameter DiT and two distilled LoRAs. It follows Meituan's recommended 8-step sampling, using DMD for the first 4 steps and OPSD for the last 4 steps; long audio is stitched into a continuous shot in segments of 93 frames (25 fps). The author measured that on an M5 Max with 64 GB, generating a 3.7-second video at 480p takes about 8 minutes, with peak memory usage around 47 GB.

Why it matters It runs locally on Apple Silicon Macs, automatically stitches long audio into a continuous shot, and includes scripts for training custom lip-sync LoRAs; however, it is memory-intensive and slow to render.

Hugs: 3 · Upvotes: 3 · Downloads: 0

License: Commercial use allowed · ComfyUI: Not available · VRAM: Not specified

Reasons

  • MIT license, consistent with Meituan's original; the DiT weights are a bit-for-bit direct conversion of Meituan's bf16 weights
  • The model card provides speed, memory, and lip-sync scores for two resolutions: 480p and 1120×832

Usage

  • git clone -b feat/avatar-1.6-continuation-lora https://github.com/fedeinfe/longcat-avatar-mlx,建 Set up a Python 3.12 virtual environment and run pip install -e ".[parity]" librosa pyloudnorm pillow ftfy, download the weights to the weights directory, then run scripts/generate.py, passing --image for the portrait, --audio for the voice, --prompt for the description, and --out for the output file

Risks

  • Supports only Apple Silicon Macs, requires 64 GB of unified memory, and the weights are 51.5 GB; a segment at 1120×832 takes about 45 minutes
  • The inference code is in the author's branch, and the merge request to the original repository has not been merged; feeding audio at 30 fps will cause lip-sync issues, so you must generate at 25 fps and then interpolate frames
  • Obtain consent from the individuals before generating videos using their portraits and voices

Open

抱脸 image-to-video 上升榜

Explore the latest AI Radar updates