Benchmark
hal-core-bench-hard
Agentic benchmarks measure how well models carry out tasks that take several steps. Scores are percentages.
All agentic benchmarksScores from HAL (Princeton).
- Models measured
- 17
- Scores recorded
- 24
- Sources
- 1
- IQ weight
- 1.36
Score distribution
- 31.9%
- Mean
- 33.3%
- Median
- 17.2%
- Std dev
- 11.1%
- 10th percentile
- 51.1%
- 90th percentile
- 8.9%
- Min
This benchmark has a weight of 1.36 for chat models, measured over 17 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.
Leaderboard
| # | Model | |
|---|---|---|
| 1 | Claude Opus 4.5· HAL (Princeton) | 77.8% |
| 2 | Claude Sonnet 4.5· HAL (Princeton) | 62.2% |
| 3 | Claude Opus 4.1· HAL (Princeton) | 51.1% |
| 4 | Claude Sonnet 4· HAL (Princeton) | 46.7% |
| 5 | Claude Sonnet 4.5high· HAL (Princeton) | 44.4% |
| 6 | Claude Opus 4.1high· HAL (Princeton) | 42.2% |
| 7 | Claude Opus 4.5high· HAL (Princeton) | 42.2% |
| 8 | Gemini 3 Pro Previewhigh· HAL (Princeton) | 40% |
| 9 | Claude Sonnet 3.7high· HAL (Princeton) | 37.8% |
| 10 | Claude Sonnet 3.7· HAL (Princeton) | 35.6% |
| 11 | o4 Minihigh· HAL (Princeton) | 35.6% |
| 12 | Claude Sonnet 4high· HAL (Princeton) | 33.3% |
| 13 | GPT 4.1· HAL (Princeton) | 33.3% |
| 14 | GPT 5medium· HAL (Princeton) | 26.7% |
| 15 | OpenAI o3medium· HAL (Princeton) | 24.4% |
| 16 | Gemini 2.5 Pro Preview 06-05· HAL (Princeton) | 22.2% |
| 17 | DeepSeek V3.1· HAL (Princeton) | 20% |
| 18 | DeepSeek V3· HAL (Princeton) | 17.8% |
| 19 | o4 Minilow· HAL (Princeton) | 17.8% |
| 20 | Claude Haiku 4.5· HAL (Princeton) | 11.1% |
| 21 | Gemini 2.0 Flash· HAL (Princeton) | 11.1% |
| 22 | GPT OSS 120B· HAL (Princeton) | 11.1% |
| 23 | GPT OSS 120Bhigh· HAL (Princeton) | 11.1% |
| 24 | DeepSeek R1· HAL (Princeton) | 8.9% |
Tokens the model generated per task, as measured by the source. Comparable within this benchmark only.
Data credits: HAL (Princeton) — cite the HAL paper