Benchmark
DABstep Hard
Agentic benchmarks measure how well models carry out tasks that take several steps. Scores are percentages.
- Models measured
- 21
- Scores recorded
- 22
- Sources
- 1
- IQ weight
- 1.80
Score distribution
- 19.1%
- Mean
- 12.4%
- Median
- 21.2%
- Std dev
- 3.4%
- 10th percentile
- 45.2%
- 90th percentile
- 0%
- Min
This benchmark has a weight of 1.80 for chat models, measured over 21 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.
Leaderboard
| # | Model | Score |
|---|---|---|
| 1 | Claude Haiku 4.5· DABstep data | 90% |
| 2 | GPT 5· DABstep data | 57.7% |
| 3 | Gemini 2.5 Pro· DABstep data | 45.2% |
| 4 | Gemini 2.5 Pro Preview 05-06· DABstep data | 41% |
| 5 | Claude 3.5 Sonnet· DABstep data | 28% |
| 6 | Claude Sonnet 4· DABstep data | 19.8% |
| 7 | DeepSeek V3· DABstep data | 16.4% |
| 8 | o4 Mini· DABstep data | 14.6% |
| 9 | Claude Sonnet 3.7· DABstep data | 13.8% |
| 10 | o3 Mini· DABstep data | 13.8% |
| 11 | GPT 4.1· DABstep data | 12.4% |
| 12 | o1· DABstep data | 11.1% |
| 13 | Gemini 2.0 Flash· DABstep data | 9.8% |
| 14 | Claude 3.5 Sonnetlatest· DABstep data | 9.3% |
| 15 | GPT-4.1 mini· DABstep data | 9% |
| 16 | Llama 4 Maverick· DABstep data | 8.7% |
| 17 | GPT 4o· DABstep data | 6.1% |
| 18 | Claude Haiku 3.5· DABstep data | 5% |
| 19 | Llama 3.3 70B Instruct· DABstep data | 3.7% |
| 20 | Qwen 2.5 Coder 32B Instruct· DABstep data | 3.4% |
| 21 | Llama 4 Scout· DABstep data | 1.9% |
| 22 | Llama 3.2 1B Instruct· DABstep data | 0% |
Data credits: DABstep (Adyen), CC BY 4.0 (licence)