phylobio/DeepSeek-V4.1-Flash-epistemic-lora

The Phylo team extracted steering vectors from a system prompt emphasizing scientific rigor, applying them to layers 14–18 and 20–28. Since the inference framework does not support steering vectors, they distilled this into a rank-1 LoRA on the down-projection of shared experts, training only the A matrix (first fitting per-token strength via ridge regression, then performing forward KL divergence against the steered model). The authors state that it significantly reduces hallucinations on AA-Omniscience and HalluLens, at the cost of increased refusals and decreased accuracy on some knowledge questions, while other benchmarks remain largely unchanged; merged full weights are also available.

The Phylo team extracted steering vectors from a system prompt emphasizing scientific rigor, applying them to layers 14–18 and 20–28. Since the inference framework does not support steering vectors, they distilled this into a rank-1 LoRA on the down-projection of shared experts, training only the A matrix (first fitting per-token strength via ridge regression, then performing forward KL divergence against the steered model). The authors state that it significantly reduces hallucinations on AA-Omniscience and HalluLens, at the cost of increased refusals and decreased accuracy on some knowledge questions, while other benchmarks remain largely unchanged; merged full weights are also available.

Why It Matters The authors found that models are overly confident when evaluating scientific evidence; this adapter makes them more cautious and less prone to fabrication. It can be directly mounted in SGLang without modifying the inference framework.

Upvotes 2 · Likes 2 · Downloads 58

Rationale

  • MIT license; the model card includes charts of changes across benchmarks and 95% confidence intervals, linking to the authors' detailed article.
  • The authors note that quantizing to FP8 after merging results in some performance loss (τ²-bench 0.82 for the adapter's 0.88), so they recommend deploying it as an adapter.

Usage

  • The authors tested this on 4 B200 GPUs using SGLang's dev-dsv41 image: download DeepSeek-V4.1-Flash-1017 and this adapter, then launch using the full command from the model card with parameters such as --enable-lora --lora-paths epistemic=/lora. When making requests, specify the model name as dsv4:epistemic and enable thinking.

Risks

  • This is adapted only for the 1017 version; the public version of vLLM does not yet support LoRA for this model, and the results in the model card and detailed article come from the authors' internal version.
  • The model becomes more conservative, more frequently deferring decisions to the user or refusing to answer; deployment requires multiple data-center-grade GPUs.

Open

Hugging Face

Explore the latest AI Radar updates