MACaQ is a calibration quantization method from Huawei CSL, accompanied by a paper. It can quantize text models into 2/3/4-bit GGUF files for running with llama.cpp, or into W4A4 formats for deployment with vLLM; it also supports quantizing diffusion models such as FLUX.2-klein, Wan2.2, and Stable Audio. The authors claim that models with 122B parameters can be quantized in just 64 minutes using 32.5 GB of VRAM. They also released MACaQ-R, which re-optimizes weight encoding while preserving the original GGUF format. In the authors' comparison of Qwen3.6-35B-A3B, the KL divergence from FP16 at the same model size is lower than that of Unsloth's equivalent version.
Why it matters The same method covers llama.cpp, vLLM, Diffusers, and PyTorch, and can improve the quality of existing GGUF files without changing their format.
Stars 234 (+25 / 24h)
Reasons
- 25 new stars in 24 hours (indicating attention only); Apache-2.0 license
- Each backend has model-specific instructions, and GGUF files produced by MACaQ-R run with standard llama.cpp
Usage
- Choose a backend (Diffusers / llama.cpp / vLLM / PyTorch) according to the README, using separate Python 3.12 environments for each; if you only want to use pre-quantized models, you can directly download the Qwen3.6-35B-A3B MACaQ-R GGUF from Hugging Face
Risks
- All quantization quality comparisons were conducted by the authors; the full MACaQ workflow requires the runtime included in the repository, not standard llama.cpp