JMNI-Labs/DeepSeek-V4.1-Flash-1017-TrellisMX-FLARE

This is a deployment package bundled by JMNI-Labs for the Flare inference engine, serving the community's DeepSeek-V4.1-Flash-1017 TrellisMX K2-Mix version with tensor parallelism on two DGX Spark units: routed experts use mixed low-bit precision of K2/K3/K4, the Engram table uses 4 bits, while attention, shared experts, MTP, vision, and tokenizer remain in their official original form. It includes 16 concurrent request slots, MTP with up to 5 verification draft tokens, a payload of approximately 259 GiB, and a default KV allocation of about 14 GiB.

This is a deployment package bundled by JMNI-Labs for the Flare inference engine, serving the community's DeepSeek-V4.1-Flash-1017 TrellisMX K2-Mix version with tensor parallelism on two DGX Spark units: routed experts use mixed low-bit precision of K2/K3/K4, the Engram table uses 4 bits, while attention, shared experts, MTP, vision, and tokenizer remain in their official original form. It includes 16 concurrent request slots, MTP with up to 5 verification draft tokens, a payload of approximately 259 GiB, and a default KV allocation of about 14 GiB.

Why it matters Provides a ready-made path for those with two DGX Spark units: multi-stream concurrency, OpenAI-compatible interface, and support for long contexts.

Upvotes 2 · Likes 2 · Downloads 0

Reasons

  • The model card clearly documents the provenance chain: official MIT weights, the specific version of the community TrellisMX package, and the kernels used

Usage

  • Install Flare built from the corresponding branch on both DGX Spark units; on the first unit, use flare download to fetch the package and start it with rank 0, then join from the second unit with rank 1; clients connect to port 8000 on the first unit

Risks

  • Requires the ds41-trellismx engine branch, which has not yet been merged into the official release; the official flare-inference cannot load it
  • The package includes only a local development signature; startup requires adding --dangerously-* parameters, and the author advises using it only on trusted hosts or switching to an official signature
  • No quality data is provided compared to the official weights

Open

Hugging Face

Explore the latest AI Radar updates