Benchmark
ARC-AGI-1
Scores are percentages. Higher is better.
- Models measured
- 16
- Scores recorded
- 26
- Sources
- 1
- IQ weight
- 2.37
Score distribution
- 90.8%
- Mean
- 94.5%
- Median
- 11.2%
- Std dev
- 86.2%
- 10th percentile
- 98.5%
- 90th percentile
- 54.8%
- Min
This benchmark has a weight of 2.37 for chat models, measured over 16 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.
Leaderboard
| # | Model | Score |
|---|---|---|
| 1 | Claude Fable 5OpenAI launch table· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 98.5% |
| 2 | Claude Fable 5System Card capability summary · adaptive thinking · max effort · default sampling · 5 trials · context ≤1M unless noted· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 98.5% |
| 3 | GPT-6 AstraOpenAI launch table· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 98.5% |
| 4 | Claude Fable 5.1OpenAI launch table· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 97.5% |
| 5 | Claude Fable 5.1System Card capability summary · adaptive thinking · max effort · default sampling · 5 trials · context ≤1M unless noted· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 97.5% |
| 6 | Claude Opus 5OpenAI launch table· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 97.5% |
| 7 | Claude Opus 5Opus 5 System Card capability summary · adaptive thinking · max effort unless noted· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 97.5% |
| 8 | Claude Opus 5System Card capability summary · adaptive thinking · max effort · default sampling · 5 trials · context ≤1M unless noted· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 97.5% |
| 9 | GPT-5.6 SolOpenAI launch table· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 97.5% |
| 10 | GPT-5.6 Solxhigh· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 97.5% |
| 11 | GPT-5.6 SolSystem Card capability summary · adaptive thinking · max effort · default sampling · 5 trials · context ≤1M unless noted· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 96.5% |
| 12 | GPT 5.5Verified · xhigh· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 95% |
| 13 | GPT-5.4 ProVerified· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 94.5% |
| 14 | Claude Opus 4.6high effort· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 94% |
| 15 | GPT 5.4Verified· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 93.7% |
| 16 | Claude Opus 4.7private validation · high effort· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 93.5% |
| 17 | Claude Opus 4.8Opus 5 System Card capability summary · adaptive thinking · max effort unless noted· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 92.5% |
| 18 | Claude Opus 4.7private validation · adaptive thinking · max effort · capability summary· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 92% |
| 19 | GPT-5.2 ProVerified· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 90.5% |
| 20 | GPT-5.2 ProVerified · xhigh· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 90.5% |
| 21 | Claude Sonnet 4.6private set · 120K thinking tokens · high effort· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 86.5% |
| 22 | GPT 5.2Verified· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 86.2% |
| 23 | GPT 5.2Verified · xhigh· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 86.2% |
| 24 | Claude Opus 4.5private validation · 64K thinking tokens· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 80% |
| 25 | Step 3.5 FlashStep 3.5 technical report · Tables 12–13 · Python tool· Reported by StepFun· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 56.5% |
| 26 | Step 3.5 FlashStep 3.5 technical report · Tables 12–13 · vanilla· Reported by StepFun· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 54.8% |