This is a model package prepared for Basalt (a local inference engine for NVIDIA Blackwell sm_120 GPUs, compatible with OpenAI and Anthropic interfaces): each file contains Unsloth-quantized DeepSeek-V4.1-Flash-1017 weights, Engram tables, visual projections, a tokenizer, and a DSpark draft model, available in two tiers: UD-IQ1_M (260 GB) and UD-IQ2_XXS (285 GB). It supports image input and a context length of up to 1,048,576 tokens; the author reports single-stream generation speeds of approximately 40–179 tok/s on an RTX 5090 plus 5060 Ti, varying by tier and content.
Why it matters Experts that do not fit in VRAM are kept in system memory, allowing consumer-grade Blackwell GPUs to run this large MoE model locally.
Upvotes 8 · Likes 8 · Downloads 0
Reasons
- The model card provides perplexity, KL divergence, and preferred token agreement rates compared to the BF16 reference
Usage
- Requires Basalt 0.3.0 or higher; download the corresponding .basalt file and load it via Basalt
Risks
- Speed and quality metrics were measured by the author on their own hardware
- Files start at 260 GB, requiring large memory and disk space; only Blackwell GPUs are supported