Independent evaluation firm Artificial Analysis released Intelligence Index v5 on 10 27: Added Cognition's private coding evaluation FrontierCode 2.0 and Terminal-Bench-Science, replaced Humanity's Last Exam with a streamlined HLE-Diamond, removed GDPval-AA, SciCode, etc.; 60% scores come from private test sets or undisclosed answers to prevent models from gaming public benchmarks. Simultaneously launched Progress Index v1 to compare model capabilities across eras.
Confidence: High (95) · Event Time 2026-10-26
Reasoning
- Many press releases and media outlets cite this index (this issue's top pick, Gemini 4 Carbon, took the top spot with 59 points); rankings shifted significantly after the version update. At the launch of v5, Claude Opus 5.5 (max) led with 58 points, followed by GPT-6 Astra 57 and GPT-6.1 Sol 55, with Gemini 4 Argon at 50.
- Open-weight models saw the largest drops: The highest-scoring Kimi K3.1 Pro (weights not yet released) scored 46 points, 12 points behind the leader (in v4.3, the gap was only 3 points); The Decoder noted it fell from rank 4 to rank 9. DeepSeek V4.1 Flash 1017 and GLM-5.3, which have released weights, both scored 32 points, below Claude Haiku 5.5's 36 points.
- The four categories each account for 25%: Knowledge Work, Coding, Science, and General. The new version is more sensitive to reasoning tiers and token usage: Claude Opus 5.5 scored 35 points at the low tier and 58 points at the highest tier, with a 10-fold difference in cost per task; the median output tokens per task rose from 3.50,000 to 5.90,000.
Usage
- The full leaderboard and methodology are available at artificialanalysis.ai; the new AA-Omniscience-Public-v1.1 public subset is on Hugging Face and can be used to self-test knowledge Q&A and hallucination rates.
- When selecting a model, note which reasoning tier is reported; score differences between tiers for the same model can exceed 20 points.
Risks
- A downside of private test sets is that external parties cannot verify question errors: Artificial Analysis revised ambiguous and erroneous questions in AA-Omniscience this time (as noted by The Decoder).
- Researcher Nathan Lambert argues that the score drop for open-source models is not just about "gaming benchmarks" but also relates to the disparity in investment across various multi-domain reinforcement learning environments; category weights are set by Artificial Analysis.