The 22 community and vendor quantized versions of V4.1-Flash-1017 are consolidated into a single entry, covering formats such as GGUF, EXL3, MLX, GSQ, and NVFP4; compared to 10-30, NVIDIA 10-30 has released an NVFP4 version, and turboderp's EXL3 has added a 2.68bpw tier.
Total Downloads: 2359927 · Total Likes: 1106
Reasons
- nvidia/DeepSeek-V4.1-Flash-1017-NVFP4: Uses Model Optimizer to convert routed experts from the source MXFP4 to NVFP4 (W4A4), and converts the Engram lookup table from FP8 to MXFP4, while maintaining original precision for attention, shared experts, vision, and MTP; the model card states approximately 400 GiB (source is approximately 475 GiB) across 48 shards, under the MIT license and available for commercial use.
- NVIDIA's six comparative tests on four GB300 GPUs using vLLM show performance nearly on par with the source weights (e.g., GPQA Diamond 92.93 vs. 92.77, Terminal-Bench 2.1 81.11 vs. 81.81), noting that expert weight conversion is lossless while Engram conversion is lossy.
- turboderp/DeepSeek-V4.1-Flash-1017-exl3 currently has 2.25bpw and 2.68bpw branches, still requiring ExLlamaV3 v1.6.6 or the dev branch; additional tiers and evaluations will be added later.
Usage
- Blackwell data center cards: Use a vLLM build with MXFP4 Engram support (vllm-project/vllm#61530), running
vllm serve nvidia/DeepSeek-V4.1-Flash-1017-NVFP4 --tensor-parallel-size 4 --tokenizer-mode deepseek\_v41 --kv-cache-dtype fp8 --reasoning-parser deepseek\_v41(see the model card for full parameters); for tool calling, add--enable-auto-tool-choice --tool-call-parser deepseek\_v41. - Single card / consumer-grade: After installing ExLlamaV3 v1.6.6, run
hf download turboderp/DeepSeek-V4.1-Flash-1017-exl3 --revision 2.68bpw; for CPU + large memory, consider unsloth GGUF or ISTA-DASLab GSQ-RCO (llama.cpp b11870 or later); for Apple silicon, look at the oQ series from mlx-community.
Risks
- The NVFP4 version only supports Blackwell; the official vLLM image cannot load it, requiring the PR mentioned above to be merged; SGLang is unverified, and MTP speculative decoding has not been tested.
- Official repository discussion #33 (in non-thinking mode, short greetings with system prompts result in garbled responses of about 47%, reproducible via the official API) remains open, with no official confirmation; turboderp's model card also notes that garbled output for short prompts is likely an issue with the model itself. Quality comparisons for other quantizations are based on the authors' own tests, and download counts only reflect popularity.