A 14.8B DNA language model from institutions such as Hugging Face, already included on 10 24: paste a DNA sequence into a standard chat, and it uses English to determine whether mutations are benign or pathogenic, identify regulatory regions within the sequence, classify the organism type, and continue writing bases. This re-inclusion is because the model card previously stated "llama.cpp / GGUF not supported," but now an official GGUF version has been released, allowing local execution via llama.cpp.
Score increase 75 · Likes 285 · Downloads 3719
Reasons
- New content compared to 10 24: The official repository HuggingFaceBio/Carbon-X-GGUF is public, offering five quantization levels: BF16, Q8_0, Q6_K, Q5_K_M, and Q4_K_M; llama.cpp supports Carbon's hybrid DNA tokenizer starting from build b11742 (PR #30825). On 10 29, the main model card also replaced the statement "llama.cpp and other GGUF runtimes are not supported" with a link to the GGUF repository.
- The official team states that the llama.cpp tokenizer is identical to the original Python version: token IDs, decoded text, and chat template rendering are the same across 1,000 benchmark prompts (1.13M tokens) and 36 DNA edge cases. In all K-quantized versions, the embedding and output layers storing DNA tokens are preserved at Q8_0.
- The official team tested each quantization level on two DNA benchmarks using llama.cpp serving: FrontierVEP original (vLLM) 70.4, BF16 69.6, Q8_0 69.1, Q4_K_M 67.8; NextBase original 39.4, Q8_0 39.7, Q4_K_M 38.6. The official team states that BF16 through Q5_K_M are all within approximately one standard error of the original, while Q4_K_M scores the lowest on both benchmarks.
Usage
- Download or update llama.cpp to build b11742 or later (desktop version from llama.app or precompiled packages from GitHub releases), then run llama serve -hf HuggingFaceBio/Carbon-X-GGUF:Q8_0, which opens both a web chat interface and an OpenAI-compatible API; for terminal use, run llama cli -hf HuggingFaceBio/Carbon-X-GGUF:Q8_0.
- The GGUF file includes the chat template and default sampling parameters, requiring no additional configuration; to align with official training and evaluation, add --chat-template-kwargs '{"enable_thinking": false}' to disable thinking.
- File sizes: Q8_0 15.8 GB (officially recommended, claimed to have scores comparable to BF16), Q6_K 12.6 GB, Q5_K_M 11.1 GB, Q4_K_M 9.6 GB (use when memory is tight), BF16 29.7 GB. When asking questions, place the reference sequence and variant sequence in the same message, including known information such as gene, strand direction, and coordinates; sequences of 6 or more consecutive A/C/G/T characters are automatically treated as DNA.
Risks
- Older versions of llama.cpp (including v0.7.0) can load these files but will incorrectly tokenize DNA without raising errors; ensure the version is b11742 or later.
- The quantized versions were tested on only two of the five benchmarks (FrontierVEP, NextBase), with single sampling per benchmark and standard errors of approximately 1.5 and 1 points; official performance data for GenomeQA and BioReason after quantization is not provided.
- Not clinically validated, and the official team explicitly states it cannot be used for medical decisions; the model is inherently weak on splice site and histone marker questions, and the quality of generated long DNA sequences degrades. The model outputs DNA wrapped in <dna>…</dna> tags, which llama.cpp preserves as-is; remove them manually if pure sequences are required.
Editor's note: The benefit of local execution is that sensitive, unpublished sequences do not need to be sent to third-party services, which is precisely the risk highlighted by the official team in the limitations section.