Skip to content

Benchmark

hal-gaia

Agentic benchmarks measure how well models carry out tasks that take several steps. Scores are percentages.

Models measured
12
Scores recorded
17
Sources
1
IQ weight
1.27

Score distribution

54.7%
Mean
57.6%
Median
14.2%
Std dev
32.7%
10th percentile
68.5%
90th percentile
29.4%
Min
29.4%74.5%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.27 for chat models, measured over 12 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model
1Claude Sonnet 4.5· HAL (Princeton)74.5%
2Claude Sonnet 4.5high· HAL (Princeton)70.9%
3Claude Opus 4.1high· HAL (Princeton)68.5%
4Claude Opus 4high· HAL (Princeton)64.9%
5Claude Opus 4.1· HAL (Princeton)64.2%
6Claude Sonnet 3.7high· HAL (Princeton)64.2%
7GPT 5medium· HAL (Princeton)62.8%
8o4 Minilow· HAL (Princeton)58.2%
9Claude Opus 4· HAL (Princeton)57.6%
10Claude Haiku 4.5· HAL (Princeton)56.4%
11Claude Sonnet 3.7· HAL (Princeton)56.4%
12o4 Minihigh· HAL (Princeton)55.8%
13GPT 4.1· HAL (Princeton)50.3%
14Gemini 2.0 Flash· HAL (Princeton)32.7%
15OpenAI o3medium· HAL (Princeton)32.7%
16DeepSeek R1· HAL (Princeton)30.3%
17DeepSeek V3· HAL (Princeton)29.4%

Tokens the model generated per task, as measured by the source. Comparable within this benchmark only.

Data credits: HAL (Princeton) — cite the HAL paper