Original title: thinkingmachines/Inkling-2-Small
A general-purpose multimodal model released by Thinking Machines that takes text and images as input and outputs text, targeting applications such as agents and tool calling, coding assistants, dialogue, and retrieval-augmented generation. The architecture consists of a 48-layer decoder with sparse expert feed-forward layers, where each token selects 8 experts out of 384, mixing local and global attention; it has 594B total parameters, 17B active parameters, and supports a maximum context length of 1M tokens. The official documentation states that pre-training, mid-training, and post-training were all completed at NVFP4 precision, so the NVFP4 checkpoint is the primary recommendation, with an MXFP8 version also available (officially claimed to be only 1–2 percentage points higher in evaluations).
Why it matters Trained natively using low-precision formats, weights are Apache licensed, and mainstream inference and post-training frameworks support it from launch; it can be deployed locally or fine-tuned by users.
Upvotes 95 · Likes 95 · Downloads 12
Reasons
- In the official evaluation table, BrowseComp (with context compression) scores 90.2%, DeepSWE 72.1%, ARC-AGI-2 81.7%, and OmniDocBench 94.6%; in the same table, DeepSeek-V4.1-Flash-1017 has an AA Index of 32.6, which is higher than its 30.6, with each model outperforming the others in different metrics.
- The deployment and training ecosystem is ready: SGLang, vLLM, TokenSpeed, and Unsloth offer local inference recipes, while Miles, SkyRL, and others support post-training; third-party providers like Together, Baseten, and Cloudflare offer APIs.
- The official documentation also lists safety evaluations (FORTRESS, StrongREJECT, etc.) and links to usage policies.
Usage
- Try it online first in the Tinker Playground, or call the API according to the Tinker Cookbook; if fine-tuning with Tinker, you also need to install tml-renderers.
- Locally: Load thinkingmachines/Inkling-2-Small (NVFP4) using the SGLang / vLLM recipes provided in the model card; use Inkling-2-Small-MXFP8 if FP8 is required.
Risks
- High hardware requirements: The NVFP4 version requires approximately 500GB of total VRAM (e.g., 2 B300 cards or 4 H200 cards), while the MXFP8 version requires about 800GB.
- The performance table is provided by the official team; scores for some comparison models come from third-party leaderboards or public splits, and evaluation settings vary.
- This entry has not been run; refer to the model card for actual performance.