SenseNova-Loop-9B: A small multimodal model that runs the middle layers twice

The official version uploaded by the SenseNova team on 11/17 (a Preview was released earlier on 10/31). The model card states approximately 9B parameters: within a 32-layer decoder, layers 4–27 share weights and are executed twice per forward pass, resulting in a total of 56 computational layers (officially equivalent to about 14.6B inference parameters). This is combined with dual residual stream mHC, and a hybrid structure featuring 3 linear attention layers every 4 layers plus 1 full attention layers. It accepts text and images as input and outputs text, with a context window of 512K, targeting complex reasoning, coding, tool calling, and multimodal tasks. Licensed under Apache-2.0.

Original title: sensenova/SenseNova-Loop-9B

The official version uploaded by the SenseNova team on 11/17 (a Preview was released earlier on 10/31). The model card states approximately 9B parameters: within a 32-layer decoder, layers 4–27 share weights and are executed twice per forward pass, resulting in a total of 56 computational layers (officially equivalent to about 14.6B inference parameters). This is combined with dual residual stream mHC, and a hybrid structure featuring 3 linear attention layers every 4 layers plus 1 full attention layers. It accepts text and images as input and outputs text, with a context window of 512K, targeting complex reasoning, coding, tool calling, and multimodal tasks. Licensed under Apache-2.0.

Why it matters Official claims indicate that agent and data analysis evaluations approach those of Qwen models three times its size; after 4-bit quantization, VRAM requirements drop to just over ten GB.

Rising score 185 · Likes 188 · Downloads 0

Reasons

  • Official claims state that the final version outperforms the Preview in all 7 evaluations shown in the chart: AutomationBench 14.33 → 36.67, Terminal-Bench 2.1 40.45 → 59.55, GAIA 37.86 → 69.90; compared to Qwen3.8-27B, AIDABench scores 65.31 vs. 65.01, and NL2Repo scores 41.36 vs. 44.60.
  • Official VRAM estimates: 4-bit weights require approximately 4.9 GB; with 8-bit KV cache, contexts of 64K / 128K / 256K require approximately 10 / 12 / 15 GiB.
  • Official experiments on loop counts show that scores are best between 3 and 4 loops within the range of 1 to 4 loops; for speed balance, the default is set to 2 loops. The model card includes animated examples of web pages, SVGs, posters, and 3D scene generation.

Usage

  • Transformers 5.18.0: Use AutoTokenizer and AutoModelForCausalLM with trust_remote_code=True, and set dtype to bfloat16; enable thinking by setting enable_thinking=True in apply_chat_template, or disable it by setting it to False.
  • Service deployment: The official LightLLM image (example --tp 4) provides an OpenAI-compatible interface; for vLLM and SGLang, you must pull unmerged PRs (vLLM #64832, SGLang #47281) and install them manually.
  • Supports MTP speculative decoding; draft model training scripts are available in GitHub OpenSenseNova/SenseSpec; fine-tuning examples are in OpenSenseNova/torchtitan.

Risks

  • Downloading weights requires logging into Hugging Face and agreeing to access terms (automatically approved); deployment commands follow the Preview version, with official updates to engine versions and configurations for the final version promised later.
  • The model card states that an official AWQ 4-bit version was provided on day one, but it was not yet visible in the public list under the sensenova namespace at the time of this review; VRAM figures are official estimates.
  • Officially listed weaknesses: missing requirements when there are many tool definitions or constraints, instability in coding and long-chain agent tasks, and complex 3D/animated web pages requiring two to three rounds of modification; all scores come from official charts and were not independently run for this review.

Open

Hugging Face Trending

Explore the latest AI Radar updates