(Repository description) A hierarchical inference engine for frontier MoE models: hot experts on the GPU (CUDA, Vulkan), warm ones in RAM, cold ones streamed from NVMe. Written in C, runs on the hardware you already own.
Stars 47253 (+802 / 24h)
Reasoning
- Added 802 stars in the last 24 hours, total stars 47253 (represents attention only).
Usage
- See the repository README for usage instructions.
Risk
- Automated draft: has not yet been reviewed by an editor.