Iris-Z: A FLUX.2 fine-tuned model that simultaneously generates images and depth maps

A generative model uploaded by speridlabs on 11 4, fine-tuned from FLUX.2-dev and FLUX.2-klein-base-9B. The same set of weights supports text-to-image generation, text-to-image with simultaneous depth output, depth estimation from photos, and image generation from depth maps. It comes in two variants: Iris-Z-big (32.2B) and Iris-Z-small (9.1B), supporting 512 and 1024 resolutions; the paper is arXiv 2611.16010.

Original title: speridlabs/iris-z

A generative model uploaded by speridlabs on 11 4, fine-tuned from FLUX.2-dev and FLUX.2-klein-base-9B. The same set of weights supports text-to-image generation, text-to-image with simultaneous depth output, depth estimation from photos, and image generation from depth maps. It comes in two variants: Iris-Z-big (32.2B) and Iris-Z-small (9.1B), supporting 512 and 1024 resolutions; the paper is arXiv 2611.16010.

Why it matters The authors claim that depth estimation accuracy surpasses specialized models like MoGe-3, while retaining the original model's image generation capabilities, making it suitable for 3D and post-production workflows requiring depth maps

Upvotes 40 · Likes 40 · Downloads 0

Reasons

  • AbsRel×100 provided by the authors (lower is better): Under the 7 dataset protocol of MoGe-3, the big model averages 3.56 and the small model 3.84, while the authors' re-run of MoGe-3 scores 3.70; under the Marigold V2 protocol, big scores 2.92 and small scores 3.11
  • The authors state that depth estimation from photos takes 4 steps on a single H200: big takes approximately 1.44 / 5.29 seconds at 512 / 1024 resolutions, and small takes approximately 0.52 / 1.92 seconds; for text-to-image generation with 50 steps, small takes approximately 38.3 seconds at 1024 resolution
  • Outputs include relative log depth (.npy), colored depth maps, and sky masks; depth map-based repainting supports three input formats: depth, parallax, and color

Usage

  • After git clone speridlabs/iris-z-sdk, run uv sync (requires Python 3.10, PyTorch 2.9.1, CUDA 13.0, and driver 580 or higher); first agree to the FLUX.2-dev terms on Hugging Face and run hf auth login (required to access its autoencoder)
  • Command line: uv run python -m iris\_z.sample depth --image photo.jpg (add --model small to use the small variant); the image, rgbd, and paint subcommands correspond to text-to-image, text-to-image with depth, and depth map-based repainting, respectively
  • Python: IrisZPipeline.from\_pretrained("speridlabs/iris-z", variant="big"); when estimating depth only, add load\_text\_encoder=False to avoid loading the text encoder; for benchmark reproduction, use seeds=range(10) and take the pixel-wise median

Risks

  • VRAM: The authors state that running all features on big requires approximately 112 GB, and small requires approximately 33 GB; consumer GPUs cannot run big
  • Weights are licensed under the FLUX non-commercial license, while the SDK code is Apache-2.0; you must also accept the gating terms for FLUX.2-dev
  • Depth scores are obtained by the authors following the paper's protocol, and some comparison models were re-run by the authors; the card does not mention ComfyUI support. This entry has not been run

Open

Hugging Face Trending

Explore the latest AI Radar updates