Mistral's multimodal MoE model, with 1T total parameters and 52B active per token, was already indexed on 10 30. This re-indexing is because a technical report (assets/technical_report.pdf) was added to the repository on 11 2, detailing training, architectural trade-offs, and deployment specifics; the model weights remain unchanged.
Rising score 503 · Likes 1500 · Downloads 5500
Reasons
- Compared to the content from 10 30, the most noteworthy addition in the report is the "agent penalty": to save VRAM, ML4 originally used sliding window attention with a 512 window, cross-layer KV sharing, and YOCO, compressing the KV cache to 1/16 of full attention. This had almost no impact on perplexity or knowledge benchmarks, but after SFT, long-context agent tasks like SWE-bench and BrowseComp degraded significantly. Therefore, a "model surgery" was performed before the second stage of pre-training to revert to near-full attention (keeping only YOCO), which was later switched to DeepSeek sparse attention.
- Training and post-training: Pre-trained for 45 days on 3,840 GB200 GPUs near Paris, totaling approximately 4150,000 GPU hours; post-training involved agents automatically generating executable environments, tasks, reference solutions, and validators covering software engineering, cybersecurity, and knowledge work; the low / medium / high reasoning effort tiers were obtained by distilling three teacher models with different length penalties into a single model.
- Officially reported scores: Terminal-Bench 4.0 41%, FrontierCode 2.0 19.7%, AA Cyber Index 50.9%; quantizing only the routed experts to NVFP4 results in a checkpoint size of 606 GB, with all benchmarks matching the FP8 version, and deployment possible on 4 GB300 GPUs.
Usage
- The report is located in assets/technical_report.pdf within the Files section of the model page. Deployment methods remain unchanged: use the vllm/vllm-openai:mistral-large-4 image for vLLM, and the lmsysorg/sglang:mistral-large-4 image for SGLang; specify low / medium / high via reasoning_effort in requests.
- If VRAM is insufficient, consider the NVFP4 version mistralai/Mistral-Large-4-1T-A52B-NVFP4 (targeted at Blackwell GPUs), or directly call Mistral API's mistral-large-4.
Risks
- The comparison tables in the report are mostly official self-tests or internal evaluations. In the same table, DeepSeek-V4.1-Flash-1017 has a higher AA Cyber Index of 56.8%; ML4's HLE-Diamond score of 17.3% and AA-Omniscience accuracy of 26.0% appear lower in the table.
- With 1T parameters, both the original FP8 version and the NVFP4 version require multiple data-center-grade GPUs and cannot run on personal devices.
- The license is the Mistral Ecosystem Open-Weight License (a modified MIT); a separate commercial license is required for Model-as-a-Service (MaaS) businesses.