Picom-code/pico-dgx-ds4

pico-dgx-ds4 is a fork of blaesus/dsgx10 that allows multiple users to share DeepSeek V4.1 Flash 1017 on a set of 4 DGX Spark units (TP4, 200G RoCE switch). The author claims to have expanded the context from 256K to 1M, added image input, and configured 16 request slots (4 slots of 1M and 12 slots of 128K). When 16 users send different requests simultaneously, throughput is approximately 380 tok/s; for single-user chat and coding, it is approximately 82 and 144 tok/s, respectively. Quality evaluations are on par with upstream.

pico-dgx-ds4 is a fork of blaesus/dsgx10 that allows multiple users to share DeepSeek V4.1 Flash 1017 on a set of 4 DGX Spark units (TP4, 200G RoCE switch). The author claims to have expanded the context from 256K to 1M, added image input, and configured 16 request slots (4 slots of 1M and 12 slots of 128K). When 16 users send different requests simultaneously, throughput is approximately 380 tok/s; for single-user chat and coding, it is approximately 82 and 144 tok/s, respectively. Quality evaluations are on par with upstream.

Why it matters When small teams build their own inference systems, concurrent throughput for multiple users is more critical than single-user speed. This fork primarily addresses that issue.

Stars 25

Reasons

  • The README lists changes item by item and includes comparison tables against upstream and SGLang

Usage

  • Requires 4 DGX Spark units and RoCE interconnect; deploy according to the README

Risks

  • Performance figures are self-tested by the author; speed-up options such as expert re-routing are lossy and disabled by default
  • High hardware requirements

Open

GitHub

Explore the latest AI Radar updates