Original title: KBlueLeaf/MOSS-Image-heal
KBlueLeaf performed a full fine-tune on MOSS-Image-Base, training only the transformer while retaining the official text encoder, VAE, and other components. The goal was to "heal" the residual stream: the author reports that the maximum value in the stream dropped from 58,411 to 3,083 (with an fp16 upper limit of 65,504), the RMS of the residual stream decreased by a factor of 7.5, and weight outliers were also suppressed, while the output remained close to the base model (velocity direction cosine 0.9983). Training involved approximately 320 ten-thousand samples and about 200 B300 GPU hours, with images sourced from approximately 100 ten-thousand danbooru images, and descriptions including tags, natural language, and GBC graph structures in various formats. Diffusers weights and ComfyUI bf16 / fp16 / int8 single files are provided.
Why it matters Leaves numerical headroom for users requiring half-precision or 8-bit quantized inference, keeping the output close to the base model
Upvotes 65 · Likes 67 · Downloads 0
Reasons
- The author states that in fp16, the base model's residual stream was only about 6% away from overflow, while the heal version leaves about 20 times more headroom, making low-precision and quantized inference more stable
- All figures and charts are included in the reports/ directory, and the method is documented publicly
- The ComfyUI single file can directly replace the model in the official MOSS-Image workflow
Usage
- ComfyUI: Place any file from comfyui/ into models/diffusion_models/, using the official MOSS-Image text encoder and VAE; use the fp16 file with --fp16-unet
- diffusers: Load the official MOSS-Image-Base pipeline, then replace the transformer with the one from this repository
Risks
- All training images are from danbooru, and the author has not separately specified performance on realistic subjects
- Only two comparison samples are provided
- This entry has not been run; refer to the model card for actual performance