Qwen3.8-27B-pi: LoRA for Reinforcement Learning Training of the pi Coding Agent

LoRA adapter (r=32) uploaded by harness-hub on 11/16, based on Qwen/Qwen3.8-27B, trained using TRL's AsyncGRPO reinforcement learning to serve as the model behind the pi coding agent for software engineering and terminal tasks. Training tasks were sourced from scale800 (750 items) and TermGrade (300 items), with rewards consisting of accuracy scores from task validators plus a small bonus for tool efficiency; this repository is the checkpoint at step 160. Licensed under Apache-2.0, with a GGUF version of the adapter also available.

Original title: harness-hub/Qwen3.8-27B-pi-adapted

LoRA adapter (r=32) uploaded by harness-hub on 11/16, based on Qwen/Qwen3.8-27B, trained using TRL's AsyncGRPO reinforcement learning to serve as the model behind the pi coding agent for software engineering and terminal tasks. Training tasks were sourced from scale800 (750 items) and TermGrade (300 items), with rewards consisting of accuracy scores from task validators plus a small bonus for tool efficiency; this repository is the checkpoint at step 160. Licensed under Apache-2.0, with a GGUF version of the adapter also available.

Why it matters Official reports indicate significantly higher task pass rates and more than a 50% reduction in tool calls; the training recipe is public, allowing adapters to be trained for other agent frameworks following this approach

Rising score 132 · Likes 133 · Downloads 0

Reasons

  • Official reports state that under pi, with one run per question and temperature 0.8: Terminal-Bench 2.1 improved from 39.5% to 55.8%, with 62% fewer tool calls per question; on the hardest 51 questions of SWE-bench Pro, performance improved from 29.4% to 62.7%, with 55% fewer tool calls
  • The model card provides confidence intervals and sign tests for paired comparisons, and separately lists the 68 Terminal-Bench questions not used for checkpoint selection (39.7% → 55.9%)
  • Usage covers attaching LoRA in vLLM, serving via transformers, using llama.cpp with GGUF adapters, macOS's Llama.app, and merging into a full model using PEFT (BF16 approx. 54 GB)

Usage

  • vLLM: vllm serve Qwen/Qwen3.8-27B --enable-lora --max-lora-rank 32 --lora-modules qwen3.8-pi=harness-hub/Qwen3.8-27B-pi-adapted, and add --mamba-cache-mode align (without this, the entire history is re-read each turn), --tool-call-parser qwen3\_xml, and disable thinking
  • pi (Node.js 22.19 or higher): Install @earendil-works/pi-coding-agent via npm, add a local service in ~/.pi/agent/models.json, and ensure the API is set to openai-completions (the adapter was not trained using the Responses API)
  • Local execution: Use llama.cpp with the Q8_0 GGUF of Qwen3.8-27B plus the F16 adapter from harness-hub/Qwen3.8-27B-pi-GGUF; lower quantization bits for the base model result in greater deviation from training-time performance

Risks

  • The repository name specified in the commands within the model card is harness-hub/Qwen3.8-27B-pi, but this address was inaccessible at the time of review; it should be changed to this repository's name, harness-hub/Qwen3.8-27B-pi-adapted
  • Training and evaluation were conducted only under the pi framework with thinking disabled; there is no data for other agent frameworks or when thinking is enabled; the GGUF version has not been re-evaluated
  • Training stopped at step 191 out of a planned 1000 steps, selecting the checkpoint from step 160; SWE-bench Pro scores were consulted during selection, so improvements in this metric may be optimistic; this entry has not been run

Open

Hugging Face Trending

Explore the latest AI Radar updates