Original title: speridlabs/iris-z
A generative model uploaded by speridlabs on 11 4, fine-tuned from FLUX.2-dev and FLUX.2-klein-base-9B. The same set of weights supports text-to-image generation, text-to-image with simultaneous depth output, depth estimation from photos, and image generation from depth maps. It comes in two variants: Iris-Z-big (32.2B) and Iris-Z-small (9.1B), supporting 512 and 1024 resolutions; the paper is arXiv 2611.16010.
Why it matters The authors claim that depth estimation accuracy surpasses specialized models like MoGe-3, while retaining the original model's image generation capabilities, making it suitable for 3D and post-production workflows requiring depth maps
Upvotes 40 · Likes 40 · Downloads 0
Reasons
- AbsRel×100 provided by the authors (lower is better): Under the 7 dataset protocol of MoGe-3, the big model averages 3.56 and the small model 3.84, while the authors' re-run of MoGe-3 scores 3.70; under the Marigold V2 protocol, big scores 2.92 and small scores 3.11
- The authors state that depth estimation from photos takes 4 steps on a single H200: big takes approximately 1.44 / 5.29 seconds at 512 / 1024 resolutions, and small takes approximately 0.52 / 1.92 seconds; for text-to-image generation with 50 steps, small takes approximately 38.3 seconds at 1024 resolution
- Outputs include relative log depth (.npy), colored depth maps, and sky masks; depth map-based repainting supports three input formats: depth, parallax, and color
Usage
- After
git clone speridlabs/iris-z-sdk, runuv sync(requires Python 3.10, PyTorch 2.9.1, CUDA 13.0, and driver 580 or higher); first agree to the FLUX.2-dev terms on Hugging Face and runhf auth login(required to access its autoencoder) - Command line:
uv run python -m iris\_z.sample depth --image photo.jpg(add--model smallto use the small variant); theimage,rgbd, andpaintsubcommands correspond to text-to-image, text-to-image with depth, and depth map-based repainting, respectively - Python:
IrisZPipeline.from\_pretrained("speridlabs/iris-z", variant="big"); when estimating depth only, addload\_text\_encoder=Falseto avoid loading the text encoder; for benchmark reproduction, useseeds=range(10)and take the pixel-wise median
Risks
- VRAM: The authors state that running all features on big requires approximately 112 GB, and small requires approximately 33 GB; consumer GPUs cannot run big
- Weights are licensed under the FLUX non-commercial license, while the SDK code is Apache-2.0; you must also accept the gating terms for FLUX.2-dev
- Depth scores are obtained by the authors following the paper's protocol, and some comparison models were re-run by the authors; the card does not mention ComfyUI support. This entry has not been run