local-inference-lab/DeepSeek-V4.1-Flash-1017-FLARE

This repository releases the DeepSeek-V4.1-Flash-1017 runtime package for Flare (NVIDIA's open-source Rust local inference runtime). Each version includes preprocessed weights, compiled kernels, captured execution graphs, and signed metadata for four-GPU tensor parallelism. It supports text and image inputs, with a maximum of 1,048,576 tokens per request, and offers an OpenAI-compatible interface. There are two platform versions: gb202 and gb10, where the latter pre-expands expert scaling to eliminate runtime conversions.

This repository releases the DeepSeek-V4.1-Flash-1017 runtime package for Flare (NVIDIA's open-source Rust local inference runtime). Each version includes preprocessed weights, compiled kernels, captured execution graphs, and signed metadata for four-GPU tensor parallelism. It supports text and image inputs, with a maximum of 1,048,576 tokens per request, and offers an OpenAI-compatible interface. There are two platform versions: gb202 and gb10, where the latter pre-expands expert scaling to eliminate runtime conversions.

Why It Matters Kernels and execution graphs are pre-compiled and require no Python, allowing local DGX or multi-GPU machines to directly launch OpenAI-compatible services.

Upvotes 2 · Likes 18 · Downloads 0

Reasons

  • The runtime package is signed by the publisher, and Flare verifies the signature before loading.

Usage

  • After installing Flare, download and run the version corresponding to your platform; files from the same version must be kept together and cannot be mixed.

Risks

  • Transformers, vLLM, SGLang, and llama.cpp cannot load this format.
  • Requires four GPUs or four devices.

Open

Hugging Face

Explore the latest AI Radar updates