An agent foundation model from Duke University, ByteDance, the University of Maryland, and the University at Buffalo, trained in just three steps: two stages of lightweight SFT followed by reinforcement learning, with no dedicated mid-training, DPO, or online distillation. The first SFT stage trains tool-free reasoning: after synthesizing difficult problems, for questions already solved by the base model, difficulty is increased through combination, parameterization, changing value ranges, and case-by-case analysis, with reference answers verified after each rewrite; the second stage uses public databases such as NASA's exoplanet archive, OurAirports, OpenAlex, UniProt, and Ensembl, where programs first filter records and compute unique reference answers, then large models generate questions, requiring the model to rely solely on search, web browsing, and Python to find evidence; finally, asynchronous GRPO is used to reinforce long-horizon tool interactions. Beyond the weights, data construction, generation, filtering, validation, prompts, and training code are all released.
Likes 39 · Code Stars 37
Reasons
- Paper results listed in the README: Axion 1.0-122B (based on Qwen3.5-122B-A10B) achieves 59.0 on the full HLE with restricted website access, 87.0 on FrontierScience Olympiad, and 66.7 on FrontierScience Research; Axion 1.0-35B (based on Qwen3.6-35B-A3B, with 3B active parameters) achieves 62.2 on SEAL-0; the authors claim it is comparable to leading commercial models on some agent benchmarks.
- Training is inexpensive: The README states that the first SFT stage uses 10,819 trajectories and the second uses 7,544 (approximately 312M tokens). The two SFT stages take approximately 90 GPU hours for 35B on 32 cards, and approximately 240 GPU hours for 122B.
- Reproducible: Training data is available on Hugging Face at Axion1-0/axion-training-data, with code under Apache-2.0 (SFT uses Swift + Megatron-FSDP, RL uses verl); the checkpoint Axion1-0/Axion-1.0-35B on Hugging Face is a reproduction following the recipe (Apache-2.0), with the model card stating that the reproduced version achieves 52.8 on HLE and 60.4 on SEAL-0, close to the paper's results; daily likes on Hugging Face are 39 and code stars are 37 (representing attention only).
Usage
- Deploy the reproduced version (the model card example uses 8 cards, adjustable based on VRAM): vllm serve Axion1-0/Axion-1.0-35B --served-model-name axion-35b --tensor-parallel-size 8 --dtype bfloat16 --max-model-len 131072 --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder, then call the OpenAI-compatible interface.
- Data generation / Evaluation: git clone https://github.com/bytedance/axion1_0.git，另建, set up a Python 3.11+ environment with python -m pip install -e ".[data,evaluation]", and generate data for both stages according to the READMEs in axion/datagen/stage1 and stage2; axion/evaluation handles data preparation and scoring for FrontierScience, HiPhO, HLE-130, and SEAL-0.
- Training: Follow axion/training/README.md to perform the two SFT stages and asynchronous GRPO; trained checkpoints must first be exported to Hugging Face format before serving with vLLM.
Risks
- The README refers to the 35B checkpoint as a community reproduction, not an official Axion checkpoint; weights for the 122B model in the paper and the SFT-only 4B / 9B variants are not available.
- Agent evaluation requires configuring an external framework to execute tool calls separately; the repository's evaluation scripts only handle data preparation and scoring. RL requires separating rollout, training, and auxiliary services into different GPU pools, resulting in a high barrier to reproduction.
- Baseline scores marked with * in the comparison table were measured by the Axion team themselves, while others are taken from official reports; all scores are as reported by the authors.