Original title: OpenMOSS-Team/MOSS-Image-SFT
Supervised fine-tuned version of the MOSS-Image series by OpenMOSS (MOSI.AI), fine-tuned from MOSS-Image-Base, with the same architecture and inference pipeline. Prompts can be natural language, tag sequences, or JSON layouts: use summary.caption and items to describe the scene, text to render, objects, and their bounding boxes (in yxyx format, centered absolute coordinates), controlling text placement and object arrangement; modifying text, descriptions, and boxes in the layout enables structured edits. The authors' comparison images claim that the SFT version has clearer text grouping and neater whiteboard layouts. The built-in VAE uses the FLUX.2 small decoder. Repository created on 10 11, not included in previous issues.
Score increase 74 · Likes 917 · Downloads 19515
Reasons
- Addresses the need for "placing specific Chinese text in images and arranging them according to a layout": the examples in the card directly write multi-line Chinese sticky note content in the prompt; JSON layouts allow text and objects to be placed at specified positions, suitable for layout-heavy images like posters, whiteboards, and infographics.
- Comprehensive ecosystem: prompt enhancement skill moss-image-prompt-enhancer (converts ideas into detailed descriptions or layout JSON, with built-in scripts for coordinate conversion only, without calling image generation services), Gradio interface, ComfyUI plugin, technical report (arXiv 2610.23374), and online Demo.
- Weights and inference code are under Apache-2.0; Hugging Face likes 917 (indicating attention level only).
Usage
- diffusers: pip install "diffusers>=0.38" "transformers>=5.3" "accelerate>=1.10" safetensors Pillow (Python 3.10+, PyTorch 2.9+, NVIDIA GPU), DiffusionPipeline.from_pretrained('OpenMOSS-Team/MOSS-Image-SFT', custom_pipeline='pipeline_moss_image', trust_remote_code=True, torch_dtype={'default': torch.bfloat16, 'vae': torch.float32}), pipe(prompt, height=1024, width=1024, num_inference_steps=50, guidance_scale=3.0, uncond_skip_layers=22).
- Default parameters: Flow-matching Euler 50 steps, CFG 3, shift 5, negative branch skips layer 22, negative prompt left empty; width and height must be multiples of 16. Layout prompts are passed using json.dumps(layout, ensure_ascii=False), and the output image size matches the canvas in the layout.
- ComfyUI: Search for MOSS-Image in ComfyUI Manager to install the plugin (integration into the official repository is ongoing), example workflows are in the comfyui/ directory of GitHub OpenMOSS/MOSS-Image; if you don't want to write layouts, you can first let an agent use the enhancement skill to generate prompts.
Risks
- Default 50 steps with two forward passes per step, slower than current few-step models; the card does not provide VRAM requirements or speed data.
- Layout generation is marked as experimental; the enhancement skill is explicitly designed for GPT-6 series agents, with unknown effectiveness for other agents.
- Includes custom pipeline code, requiring trust_remote_code; the card only contains one set of Base vs. SFT comparison images, without quantitative evaluation tables.