A preview multimodal model released by SenseTime on 10 30, which takes text and images as input and outputs text, targeting long-context and agentic tasks (reading documents, writing code, calling external tools); structurally, it shares weights for layers 4–27 and runs them twice, using 32 layers of weights to perform 56 layers of computation, and manages information flow with two mHC residual streams.
Upvotes 70 · Likes 70 · Downloads 12
Reasons
- The approach is "adding depth without adding parameters": approximately 9B stored parameters and approximately 14.6B inference compute parameters; the decoder alternates between three layers of linear attention and one layer of full attention, with a maximum training sequence length of 262,144 tokens. MTP acceleration is already supported, while draft models for DSpark and DFlash2 are still in training.
- The model card showcases examples of dynamic design short videos, PPT design, and data analysis (officially noted as produced by its Raccoon Work product). The comparison charts cover eight agentic and office-related benchmarks, including BIRD-Dev, Terminal Bench 2.1, AutomationBench, and RaccoonPPTBench.
- Apache-2.0 license, with SFT / DPO / MTP fine-tuning scripts included (single machine, eight GPUs); this model was not included in previous editions.
Usage
- Transformers: pip install "transformers==5.18.0" accelerate safetensors (officially verified environment: PyTorch 2.11.0 + CUDA 13.0). Load with AutoModelForCausalLM using trust_remote_code=True and bfloat16 on a single GPU; toggle thinking mode via enable_thinking=True / False in apply_chat_template.
- Serving: LightLLM official image example with four GPUs (--tp 4), OpenAI-compatible /v1/chat/completions; SGLang support requires manually pulling and compiling the branch from PR 47281.
- Fine-tuning: git clone OpenSenseNova/torchtitan, then run SFT using examples/train/SenseNova-Loop-9B-Preview/mhc2_loop2_titan.sh.
Risks
- Preview version: The technical report is yet to be released. The model card states that details of the Transformers inference implementation are "to be announced," and SGLang support is only available in an unmerged PR.
- Performance metrics are only shown in comparison charts; there are no textual scores in the model card, so this column cannot verify specific numbers. The examples are from the official's own product.
- The loop structure requires inference implementations to strictly reproduce the loop range and mHC routing; third-party quantized or converted versions may not be compatible.