A pre-packaged V4.1-Flash-1017 for Flare, NVIDIA's open-source Rust inference runtime: weights, compiled CUDA kernels, and captured execution graphs are bundled in a signed package, enabling 4-card tensor parallelism to directly launch an OpenAI-compatible service with support for image-text inputs and a 1,048,576-token context window.
Upvotes 4 · Likes 16 · Downloads 0
Rationale
- Two hardware-specific versions: gb202 targets 4 RTX PRO 6000 Blackwell 96 GB GPUs on a single machine (approx. 515 GB), while gb10 targets 4 DGX Spark units (approx. 490 GB).
- No Python required, no runtime compilation; publisher signatures are verified before loading. The author claims up to 32 concurrent requests per package, with adaptive MTP verification.
- The model card was updated at 10-27; the repository was created at 10-18.
Usage
- Run
hf download local-inference-lab/DeepSeek-V4.1-Flash-1017-FLARE --revision gb202 --include '\*.safetensors' 'publisher.json' --local-dir ./models, then use theghcr.io/local-inference-lab/flare:releaseimage to run/models/model.safetensors --gpus 0,1,2,3(see the README in the gb202 branch for full parameters). - The API is available at
/v1with the model nameDeepSeek-V4.1-Flash; DGX Spark users should refer to the gb10 branch and install flare via the signed APT package.
Risks
- Can only be loaded by Flare; Transformers, vLLM, SGLang, and llama.cpp cannot read it. The gb202 example stores the Engram table in memory, requiring at least 256 GiB of host RAM.
- The model card states that comprehensive accuracy evaluations have not been performed; the packager is local-inference-lab, not DeepSeek officially.