Community quantized/packaged versions of V4.1-Flash-1017 are consolidated into a single entry, currently totaling 24 versions, covering formats such as GGUF (unsloth, ISTA DASLab, etc.), EXL3, MLX, and NVIDIA NVFP4. Compared to the previous inclusion (10-31), the new addition within this window is the GSQ-RCO Coder released by ISTA DASLab on 11-02: based on its IQ2_XXS quantized version, 192 routed experts are removed from the 384 experts per layer, and the remaining experts are selected based on calibration data for code, agent tool calling, long context, and vision.
Total Downloads 3296957 · Total Likes 1464
Rationale
- The author states: The Coder's Transformer weights are 72.12 GB (the unpruned IQ2_XXS version is 138.2 GB), which translates to approximately 1.05 bits per parameter based on the original model's parameter count; the two Engram lookup tables, totaling 110.6 GB, are shared with the unpruned version and are read line-by-line from disk per token, allowing them to remain non-resident in memory, resulting in a total download size of 182.72 GB.
- The code benchmarks provided by the author (reasoning effort 50): SWE-bench Verified drops from 81.20 in the original model to 77.00 (retaining 94.8%), and LiveCodeBench v6 drops from 89.71 to 88.57 (retaining 98.7%); experts are jointly selected using the RCO method based on whole-model KL divergence, rather than by scoring importance per layer and keeping the top-ranked ones, and the list of retained experts per layer is publicly available in the repository.
- Other branches in the summary remain unchanged; the Coder is the only new branch appearing within this window, and it is the first time this column has seen a V4.1 compressed version that prunes experts based on usage purpose.
Usage
- llama.cpp requires version b11870 or newer: hf download ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-GGUF --include "IQ2_XXS/*" "*Engram*", then link the Engram files as the 6th shard (the card provides instructions for ln -s and Windows mklink /H), point llama-cli -m to the 1th shard, and add -ngl 99.
- If you have already downloaded the unpruned GSQ-RCO version, the Engram files and the vision projection mmproj are identical, so there is no need to re-download them; add --mmproj for image processing, and use --reasoning on --reasoning-effort 50 for reasoning effort.
- For general conversation without a focus on code, the author recommends using the unpruned GSQ-RCO version; for other formats (unsloth GGUF, EXL3, MLX, NVFP4), refer to the respective branch pages in the summary.
Risks
- The author explicitly states this is an experimental release: removing half of the experts inevitably results in capability loss, and the degradation of capabilities outside of code/tool calling/long context/vision is the cost of this method; the card only provides two code benchmarks; the runtime steps and VRAM figures for the single-card Strata engine are still marked as TBD in the card.
- All scores are self-tested by the author at reasoning effort 50, with no third-party reproduction; it still requires a memory/VRAM combination capable of holding approximately 72 GB of weights and approximately 183 GB of disk space, and placing the Engram tables on an external drive will slow down prefilling.