jesdga/DeepSeek-V4.1-Flash-1017-Basalt

This is a model package prepared for Basalt (a local inference engine for NVIDIA Blackwell sm_120 GPUs, compatible with OpenAI and Anthropic interfaces): each file contains Unsloth-quantized DeepSeek-V4.1-Flash-1017 weights, Engram tables, visual projections, a tokenizer, and a DSpark draft model, available in two tiers: UD-IQ1_M (260 GB) and UD-IQ2_XXS (285 GB). It supports image input and a context length of up to 1,048,576 tokens; the author reports single-stream generation speeds of approximately 40–179 tok/s on an RTX 5090 plus 5060 Ti, varying by tier and content.

This is a model package prepared for Basalt (a local inference engine for NVIDIA Blackwell sm_120 GPUs, compatible with OpenAI and Anthropic interfaces): each file contains Unsloth-quantized DeepSeek-V4.1-Flash-1017 weights, Engram tables, visual projections, a tokenizer, and a DSpark draft model, available in two tiers: UD-IQ1_M (260 GB) and UD-IQ2_XXS (285 GB). It supports image input and a context length of up to 1,048,576 tokens; the author reports single-stream generation speeds of approximately 40–179 tok/s on an RTX 5090 plus 5060 Ti, varying by tier and content.

Why it matters Experts that do not fit in VRAM are kept in system memory, allowing consumer-grade Blackwell GPUs to run this large MoE model locally.

Upvotes 8 · Likes 8 · Downloads 0

Reasons

  • The model card provides perplexity, KL divergence, and preferred token agreement rates compared to the BF16 reference

Usage

  • Requires Basalt 0.3.0 or higher; download the corresponding .basalt file and load it via Basalt

Risks

  • Speed and quality metrics were measured by the author on their own hardware
  • Files start at 260 GB, requiring large memory and disk space; only Blackwell GPUs are supported

Open

Hugging Face

Explore the latest AI Radar updates