Benchmark
Agents' Last Exam · Overall Score
Agentic benchmarks measure how well models carry out tasks that take several steps. Higher scores are better.
- Models and systems measured
- 17
- Scores recorded
- 50
- Sources
- 1
- IQ weight
- 0.79
Score distribution
- 49
- Mean
- 51.2
- Median
- 7.4
- Std dev
- 38.5
- 10th percentile
- 56.4
- 90th percentile
- 29.6
- Min
This benchmark has a weight of 0.79 for chat models, measured over 6 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.
Leaderboard
| # | Model or system | Score |
|---|---|---|
| 1 | GPT-6 AstraOpenAI launch table · highest published effort· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 59.3 |
| 2 | GPT-6 Astramax effort · $6.2337 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 59.3 |
| 3 | GPT-6 Astraxhigh effort · $5.397 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 58.3 |
| 4 | GPT-6 Astrahigh effort · $4.6396 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 57.8 |
| 5 | GPT-6 Astramedium effort · $4.1046 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 57.6 |
| 6 | GPT-6 Solmax effort · $2.9313 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 56.4 |
| 7 | Claude Opus 5high effort · $7.2917 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 55.9 |
| 8 | Claude Opus 5xhigh effort · $10.0226 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 55.5 |
| 9 | Claude Opus 5OpenAI launch table · highest published effort· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 55.5 |
| 10 | GPT-6 Solxhigh effort · $1.668 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 55.4 |
| 11 | GPT-5.6 Solxhigh effort · $5.0764 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 53.6 |
| 12 | GPT-5.6 SolOpenAI launch table · highest published effort· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 53.6 |
| 13 | GPT-5.6 SolQwen表:Overall Score· Reported by Qwen· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 53.6 |
| 14 | GPT-6 Astralow effort · $3.0726 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 53.4 |
| 15 | GPT-6 Solmedium effort · $1.2664 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 53.1 |
| 16 | Claude Opus 5medium effort · $5.2907 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 53 |
| 17 | GPT-5.6 Solmax effort · $7.1322 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 52.8 |
| 18 | GPT-5.6 SolOpenAI GPT-5.6 launch table· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 52.7 |
| 19 | Claude Opus 5max effort · $9.7583 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 52.7 |
| 20 | GPT-6 Solhigh effort · $1.5302 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 52.6 |
| 21 | Qwen3.8 MaxQwen表:Overall Score· Reported by Qwen· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 52.4 |
| 22 | GPT-5.6 Solhigh effort · $3.7874 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 52.4 |
| 23 | GPT-5.6 Solmedium effort · $3.4126 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 52.1 |
| 24 | Claude Opus 5low effort · $4.0619 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 51.9 |
| 25 | Qwen 3.8 Flash NextQwen3.8-Flash-Next release/model card · Overall Score· Reported by Qwen· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 51.2 |
| 26 | GPT-6 Lunamax effort · $0.1545 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 50.9 |
| 27 | GPT-5.6 TerraOpenAI GPT-5.6 launch table· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 50.4 |
| 28 | GPT-5.6 Lunamax effort · $2.5698 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 50.4 |
| 29 | GPT-5.6 LunaOpenAI GPT-5.6 launch table· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 50.3 |
| 30 | GPT-5.6 Lunaxhigh effort · $1.5468 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 49.4 |
| 31 | Claude Fable 5OpenAI launch table · highest published effort· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 48.7 |
| 32 | GPT-6 Sollow effort · $0.8641 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 48.7 |
| 33 | GPT-6 Lunaxhigh effort · $0.1105 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 47.9 |
| 34 | GPT 5.5OpenAI GPT-5.6 launch table· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 46.9 |
| 35 | GPT-6 Lunamedium effort · $0.1054 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 46.8 |
| 36 | GPT-5.6 Lunahigh effort · $0.8389 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 46.1 |
| 37 | Claude Opus 4.8OpenAI GPT-5.6 launch table· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 45.2 |
| 38 | Claude Opus 4.8Qwen表:Overall Score· Reported by Qwen· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 45.1 |
| 39 | GPT-5.6 Sollow effort · $1.6324 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 45.1 |
| 40 | GPT-6 Lunahigh effort · $0.1117 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 43.6 |
| 41 | Qwen 3.8 27BQwen3.8-27B model card · Overall Score· Reported by Qwen· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 42.9 |
| 42 | Claude Fable 5OpenAI GPT-5.6 launch table· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 40.5 |
| 43 | GPT-5.6 Lunamedium effort · $0.375 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 38.5 |
| 44 | GPT-6 Lunalow effort · $0.0247 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 36.3 |
| 45 | Gemini 3.1 Pro PreviewOpenAI GPT-5.6 launch table· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 32.1 |
| 46 | Qwen3.7 MaxQwen表:Overall Score· Reported by Qwen· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 31.1 |
| 47 | GPT-5.6 Lunalow effort · $0.1761 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 31 |
| 48 | MiniMax M3.1 FlashMiniMax M3.1 Flash official-weights model-card run · reasoning effort, sampling, and run count not stated for this benchmark · official ALE Overall full-set leaderboard result · CLI-only runs excluded· Reported by MiniMax· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 29.6 |
Not ranked
The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.
| Model or system | Score |
|---|---|
| Claude Fable 5 with Opus 4.8 fallbackadaptive effort · $15.2285 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 41.3 |
| Claude Fable 5 with Opus 4.8 fallbackxhigh effort · $28.5542 average cost/task· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 48.7 |