pico-dgx-ds4 is a fork of blaesus/dsgx10 that allows multiple users to share DeepSeek V4.1 Flash 1017 on a set of 4 DGX Spark units (TP4, 200G RoCE switch). The author claims to have expanded the context from 256K to 1M, added image input, and configured 16 request slots (4 slots of 1M and 12 slots of 128K). When 16 users send different requests simultaneously, throughput is approximately 380 tok/s; for single-user chat and coding, it is approximately 82 and 144 tok/s, respectively. Quality evaluations are on par with upstream.
Why it matters When small teams build their own inference systems, concurrent throughput for multiple users is more critical than single-user speed. This fork primarily addresses that issue.
Stars 25
Reasons
- The README lists changes item by item and includes comparison tables against upstream and SGLang
Usage
- Requires 4 DGX Spark units and RoCE interconnect; deploy according to the README
Risks
- Performance figures are self-tested by the author; speed-up options such as expert re-routing are lossy and disabled by default
- High hardware requirements