A one-click package porting the Xolotl / Strata engines from Linux to Windows: runs V4.1-Flash-1017 (with 2-bit expert quantization) on a standard PC equipped with an RTX GPU of 12 GB or more, 128 GB or more of RAM, and a fast NVMe drive, providing a local interface compatible with OpenAI and Anthropic, with a maximum context length of 1M.
Star count: 28
Reasons
- Double-clicking Start DeepSeek.bat will incrementally install Python, pre-compiled engines selected based on the CPU, and model files (downloads are resumable and verified individually via SHA-256); once installed, it automatically opens the chat page, eliminating the need for manual compilation.
- The author reports generation speeds of approximately 41 tok/s (with 192 GB RAM) / 33 tok/s (with 128 GB RAM) on an RTX 3090 + NVMe setup, and long-prompt reading speeds of approximately 610 / 410 tok/s; with precise routing, the token-by-token top-choice consistency rate with the official DeepSeek API is 91.4% (general) / 92.1% (code and math), all based on the author's self-tests.
- By default, it uses the ds4 version of Q2 GGUF and employs speculative decoding draft models derived from the official MTP headers; Choose Model.bat allows switching to quantizations from ISTA-DASLab's GSQ-RCO or unsloth, with the author recommending UD-IQ1_S for 128 GB RAM.
- Adaptations have been made for Agents: up to 8 conversation segments are cached in memory, eliminating the need to re-read the entire prompt when switching; supports tools, forced tool_choice, and JSON output.
Usage
- Requires Windows 10/11 64-bit, an NVIDIA RTX GPU with 12 GB or more of VRAM (driver 580+, GTX 10 series and older are not supported), ≥128 GB RAM, and approximately 385 GB of space on an NVMe drive; place the repository in a short path (e.g., C:\AI\Delta) and double-click Start DeepSeek.bat.
- Clients connect to http://127.0.0.1:8080/v1（OpenAI）或 /v1/messages (Anthropic); the API Key can be any value, and the model name is deepseek-v4.1-flash-1017-ds4-q2; if issues arise, use Start DeepSeek (debug).bat, which packages logs excluding conversation content.
Risks
- The initial download is approximately 375 GB; the model continuously reads from the SSD during runtime, making SATA drives, mechanical hard drives, and USB drives unsuitable; the first few minutes after a restart are noticeably slower, with the author measuring approximately 17 tok/s on a 12 GB GPU, and concurrent requests are processed sequentially.
- The weights are community-quantized versions rather than the official originals, and both speed and consistency rates are measured by the author on their own machine; by default, it only listens on localhost, and an api_key must be set to use it over a local network.