Skip to content

Benchmark

BALROG NetHack

Agentic benchmarks measure how well models carry out tasks that take several steps. Scores are percentages.

Models measured
37
Scores recorded
38
Sources
1
IQ weight
1.76

Score distribution

1.6%
Mean
0.6%
Median
2.6%
Std dev
0%
10th percentile
3.3%
90th percentile
0%
Min
0%13.2%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.76 for chat models, measured over 37 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT-6 Astramax· Epoch AI Benchmarking Hub data13.2%
2Claude Opus 5max· Epoch AI Benchmarking Hub data7.3%
3Gemini 3 Pro Preview· Epoch AI Benchmarking Hub data6.8%
4Gemini 3 Flash· Epoch AI Benchmarking Hub data4%
5GPT-5.6 Terramax· Epoch AI Benchmarking Hub data3.3%
6GPT-5.6 Solmax· Epoch AI Benchmarking Hub data3.2%
7Gemini 3.1 Pro Preview· Epoch AI Benchmarking Hub data3%
8Claude Opus 4.5· Epoch AI Benchmarking Hub data2.4%
9Claude Opus 4.564k· Epoch AI Benchmarking Hub data2.4%
10Grok 4· Epoch AI Benchmarking Hub data1.8%
11Gemini 2.5 Pro· Epoch AI Benchmarking Hub data1.7%
12GPT-5.6 Lunamax· Epoch AI Benchmarking Hub data1.7%
13Grok 3beta· Epoch AI Benchmarking Hub data1.6%
14DeepSeek R1· Epoch AI Benchmarking Hub data1.4%
15GPT 5minimal· Epoch AI Benchmarking Hub data1.3%
16Claude Haiku 3.5· Epoch AI Benchmarking Hub data1.2%
17Gemini 2.5 Flash· Epoch AI Benchmarking Hub data0.8%
18DeepSeek R1 Distill Qwen 32B· Epoch AI Benchmarking Hub data0.7%
19Claude 3.5 Sonnet· Epoch AI Benchmarking Hub data0.6%
20Reka Flash 3· Epoch AI Benchmarking Hub data0.6%
21gemini-1.5-pro-002· Epoch AI Benchmarking Hub data0.4%
22GPT 4o· Epoch AI Benchmarking Hub data0.4%
23Llama 3.3 70B Instruct· Epoch AI Benchmarking Hub data0.4%
24Llama 3.1 70B Instruct· Epoch AI Benchmarking Hub data0.3%
25Mistral Nemo· Epoch AI Benchmarking Hub data0.3%
26Qwen2.5 72B Instruct· Epoch AI Benchmarking Hub data0.3%
27Claude Haiku 4.5· Epoch AI Benchmarking Hub data0%
28gemini-1.5-flash-002· Epoch AI Benchmarking Hub data0%
29GPT-4o mini· Epoch AI Benchmarking Hub data0%
30Llama 3.1 8B Instruct· Epoch AI Benchmarking Hub data0%
31Llama 3.2 11B Vision Instruct· Epoch AI Benchmarking Hub data0%
32Llama 3.2 1B Instruct· Epoch AI Benchmarking Hub data0%
33Llama 3.2 3B· Epoch AI Benchmarking Hub data0%
34Llama 3.2 90B Vision Instruct· Epoch AI Benchmarking Hub data0%
35Phi 4· Epoch AI Benchmarking Hub data0%
36Qwen2 VL 7b Instruct· Epoch AI Benchmarking Hub data0%
37Qwen2-VL (72B) Instruct· Epoch AI Benchmarking Hub data0%
38Qwen2.5 7B Instruct· Epoch AI Benchmarking Hub data0%

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence)