A prompt rewriting model fine-tuned by the LightX2V team using Qwen3.8-27B, specifically designed to write prompts for MiniMax-H3's multi-reference generation (Ref2VA): it takes a brief requirement, target duration, and numbered reference images, videos, and audio as input, then outputs storyboard-level long prompts following the attached writing guidelines. The output is divided into six sections: subject definition, summary, reference retention analysis, shot-by-shot description, ambient sound, and background music. Uploaded on 10/29, it was not included in previous issues.
Rising score 58 · Likes 59 · Downloads 12763
Reasons
- MiniMax-H3's reference mode requires clearly specifying relationships such as "the person in image 1," "the structure of video 2," and "audio 1 as background music." This model automatically generates structured prompts with <Subject N>, <Picture N>, <Video N>, and <Audio N> tags according to the guidelines, eliminating the need for manual writing.
- The repository includes complete vLLM service scripts, invocation scripts, and writing guidelines (VIDEO_PROMPT_WRITING_GUIDE_ref_en.md). Examples cover scenarios such as two people shaking hands, merging two video segments into one, and a person in an image dancing to music.
- The same team has also released a 9B version (Qwen3.5-9B) and an FL2VA version that uses only text or first/last frames.
Usage
- Run pip install vllm requests pillow qwen-vl-utils. Set MODEL_PATH in vllm_server.sh, then run bash vllm_server.sh (defaults to single GPU, 32768 context length, thinking enabled). Install ffmpeg and ffprobe if using reference videos.
- Fill in PROMPT, DURATION, and absolute paths for IMAGES / VIDEOS / AUDIOS in post_vllm.sh. Use image 1, video 1, and audio 1 in the prompt to correspond with the numbers, then run bash post_vllm.sh. Results are saved in infer_outputs/result.json; add --dry_run to generate requests without invoking them.
- Paste the output prompt into your own MiniMax-H3 Ref2VA workflow.
Risks
- The model card is brief: it does not specify a license, nor does it provide generation comparisons before and after rewriting or any evaluations. The quality of the rewriting cannot be judged by this column.
- BF16 weights are approximately 55 GB. The examples run on a single GPU, requiring a graphics card with large VRAM; if VRAM is insufficient, consider the 9B version (this column only reviewed the beginning of its model card).
- Reference audio is passed only via tags; the model does not listen to the audio content. Only 8 frames are extracted from each reference video segment.