Original title: BAAI/AREX-2
A 270-billion-parameter long-horizon agent model from the Beijing Academy of Artificial Intelligence (BAAI), based on Qwen3.8-27B (dense, multimodal), with a context window of 262,144 tokens. During testing, it improves a solution over multiple rounds: proposing, measuring, reflecting, and modifying, using scores, logs, error messages, and execution times to decide what to change next. The training data consists of tasks with directly verifiable outcomes, such as machine learning engineering and algorithmic programming, combined with the original AREX deep research data; officially, this self-improvement capability transfers to deep research without additional search trajectories. Official scores: Frontier-CS agent track 70.7, MLE-Lite medal rate 81.8, BrowseComp 84.0, GAIA 92.2, DeepSearchQA 93.8, HLE text subset 52.6.
Why it matters Turning additional test rounds into better solutions, with capabilities learned in programming and machine learning engineering transferring to deep research
Score increase 50 · Likes 205 · Downloads 2918
Reasons
- Targeted at tasks requiring multiple rounds of trial and error: machine learning engineering, algorithmic competition programming, and tool-augmented deep research
- In the official table, among open models under 40B, it ranks highly on BrowseComp, GAIA, and DeepSearchQA, and outperforms closed-source models listed in the table on MLE-Lite
- The GitHub repository VectorSpaceLab/AREX-2 releases runners and dataset definitions for six evaluation tracks, allowing re-testing under identical settings
Usage
- transformers: AutoModelForMultimodalLM.from_pretrained("BAAI/AREX-2", dtype=torch.bfloat16, device_map="auto"), requires a newer version of transformers supporting Qwen3.8
- Re-testing: After running pip install -e '.[research]', use python3 evaluate.py BrowseComp --n 10 to specify the model endpoint and judge model endpoint for execution
Risks
- The model card only provides transformers dialogue examples; running the full multi-round improvement or deep research workflow requires setting up your own tools and execution environment
- Scores come from the official paper and model card; some comparison model scores are taken from their respective reports; the HLE column includes both full sets and text subsets
- This entry has not been run; refer to the model card for actual performance