Skip to content

Benchmark

BALROG Crafter

Agentic benchmarks measure how well models carry out tasks that take several steps. Scores are percentages.

Models measured
37
Scores recorded
39
Sources
1
IQ weight
1.19

Score distribution

34.9%
Mean
31.2%
Median
16.6%
Std dev
15.9%
10th percentile
55.9%
90th percentile
6.4%
Min
6.4%76.8%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.19 for chat models, measured over 37 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT-6 Astramax· Epoch AI Benchmarking Hub data76.8%
2Claude Opus 5max· Epoch AI Benchmarking Hub data68.2%
3Gemini 3 Pro Preview· Epoch AI Benchmarking Hub data57.3%
4Grok 4· Epoch AI Benchmarking Hub data57.3%
5GPT-5.6 Solmax· Epoch AI Benchmarking Hub data55.9%
6GPT-5.6 Lunamax· Epoch AI Benchmarking Hub data55.5%
7Gemini 2.5 Pro· Epoch AI Benchmarking Hub data55%
8Gemini 3.1 Pro Preview· Epoch AI Benchmarking Hub data55%
9GPT-5.6 Terramax· Epoch AI Benchmarking Hub data55%
10Claude Opus 4.5· Epoch AI Benchmarking Hub data49.5%
11Claude Opus 4.564k· Epoch AI Benchmarking Hub data48.6%
12Gemini 3 Flash· Epoch AI Benchmarking Hub data45%
13Gemini 2.5 Flash· Epoch AI Benchmarking Hub data40%
14GPT 5minimal· Epoch AI Benchmarking Hub data39.1%
15DeepSeek R1· Epoch AI Benchmarking Hub data36.4%
16Reka Flash 3· Epoch AI Benchmarking Hub data33.6%
17GPT 4o· Epoch AI Benchmarking Hub data33.1%
18Claude 3.5 Sonnet· Epoch AI Benchmarking Hub data32.7%
19Llama 3.2 90B Vision Instruct· Epoch AI Benchmarking Hub data31.7%
20Llama 3.1 70B Instruct· Epoch AI Benchmarking Hub data31.2%
21gemini-1.5-pro-002· Epoch AI Benchmarking Hub data30.2%
22Llama 3.3 70B Instruct· Epoch AI Benchmarking Hub data28.6%
23Mistral Nemo· Epoch AI Benchmarking Hub data27.7%
24Qwen2.5 72B Instruct· Epoch AI Benchmarking Hub data27.3%
25Claude Haiku 3.5· Epoch AI Benchmarking Hub data26.4%
26Llama 3.2 11B Vision Instruct· Epoch AI Benchmarking Hub data26.2%
27Llama 3.1 8B Instruct· Epoch AI Benchmarking Hub data25.5%
28Grok 3beta· Epoch AI Benchmarking Hub data25%
29Claude Haiku 4.51k· Epoch AI Benchmarking Hub data24.1%
30Claude Haiku 4.5· Epoch AI Benchmarking Hub data24.1%
31Qwen2-VL (72B) Instruct· Epoch AI Benchmarking Hub data22.7%
32gemini-1.5-flash-002· Epoch AI Benchmarking Hub data20%
33Llama 3.2 3B· Epoch AI Benchmarking Hub data17.3%
34Qwen2.5 7B Instruct· Epoch AI Benchmarking Hub data16.4%
35GPT-4o mini· Epoch AI Benchmarking Hub data15.9%
36DeepSeek R1 Distill Qwen 32B· Epoch AI Benchmarking Hub data15%
37Phi 4· Epoch AI Benchmarking Hub data13.6%
38Llama 3.2 1B Instruct· Epoch AI Benchmarking Hub data12.7%
39Qwen2 VL 7b Instruct· Epoch AI Benchmarking Hub data6.4%

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence)