Skip to content

Benchmark

hal-tau-bench-airline

Agentic benchmarks measure how well models carry out tasks that take several steps. Scores are percentages.

Models measured
10
Scores recorded
15
Sources
1
IQ weight
0.64

Score distribution

44.1%
Mean
44%
Median
10.2%
Std dev
28%
10th percentile
56%
90th percentile
22%
Min
22%56%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.64 for chat models, measured over 10 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model
1Claude Sonnet 3.7· HAL (Princeton)56%
2o4 Minihigh· HAL (Princeton)56%
3Claude Opus 4.1· HAL (Princeton)54%
4OpenAI o3medium· HAL (Princeton)54%
5Claude Opus 4.1high· HAL (Princeton)52%
6Claude Sonnet 3.7high· HAL (Princeton)52%
7GPT 5medium· HAL (Princeton)48%
8Claude Opus 4· HAL (Princeton)44%
9Claude Opus 4high· HAL (Princeton)44%
10DeepSeek V3· HAL (Princeton)44%
11DeepSeek R1· HAL (Princeton)36%
12GPT 4.1· HAL (Princeton)36%
13o4 Minilow· HAL (Princeton)36%
14Gemini 2.0 Flashhigh· HAL (Princeton)28%
15Gemini 2.0 Flash· HAL (Princeton)22%

Tokens the model generated per task, as measured by the source. Comparable within this benchmark only.

Data credits: HAL (Princeton) — cite the HAL paper