This repository releases the DeepSeek-V4.1-Flash-1017 runtime package for Flare (NVIDIA's open-source Rust local inference runtime). Each version includes preprocessed weights, compiled kernels, captured execution graphs, and signed metadata for four-GPU tensor parallelism. It supports text and image inputs, with a maximum of 1,048,576 tokens per request, and offers an OpenAI-compatible interface. There are two platform versions: gb202 and gb10, where the latter pre-expands expert scaling to eliminate runtime conversions.
Why It Matters Kernels and execution graphs are pre-compiled and require no Python, allowing local DGX or multi-GPU machines to directly launch OpenAI-compatible services.
Upvotes 2 · Likes 18 · Downloads 0
Reasons
- The runtime package is signed by the publisher, and Flare verifies the signature before loading.
Usage
- After installing Flare, download and run the version corresponding to your platform; files from the same version must be kept together and cannot be mixed.
Risks
- Transformers, vLLM, SGLang, and llama.cpp cannot load this format.
- Requires four GPUs or four devices.