Skip to content

Benchmark

Arena Agent Bash Reliability

Agentic benchmarks measure how well models carry out tasks that take several steps. Higher scores are better.

Models measured
62
Scores recorded
68
Sources
1
IQ weight
1.47

Score distribution

-0.001
Mean
0.015
Median
0.105
Std dev
-0.129
10th percentile
0.105
90th percentile
-0.491
Min
-0.4910.142
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.47 for chat models, measured over 62 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Sonnet 5High· LMArena data0.142
2Claude Opus 5.5Max· LMArena data0.14
3Claude Fable 5.2Max· LMArena data0.131
4GPT-6.1 TerraMax· LMArena data0.127
5Claude Sonnet 5.5Max· LMArena data0.119
6GPT-6.1 TerraMedium· LMArena data0.11
7Claude Opus 5.5High· LMArena data0.108
8GPT-5.6 SolxHigh· LMArena data0.105
9GPT-6 SolMax· LMArena data0.0997
10Claude Fable 5.1Max· LMArena data0.0927
11Grok 4.6xHigh· LMArena data0.0914
12Claude Fable 5High· LMArena data0.0899
13Gemini 3.8 FlashHigh· LMArena data0.0885
14Claude Opus 5Max· LMArena data0.0874
15GPT-6 LunaMax· LMArena data0.0833
16Grok 4.7xHigh· LMArena data0.082
17GPT-6.2 AstraMax· LMArena data0.0769
18Muse Wonder FullMax· LMArena data0.0731
19Muse EmberMax· LMArena data0.0664
20Claude Haiku 5.5High· LMArena data0.0655
21GPT-6 AstraMax· LMArena data0.0649
22Kimi K3.1 ProMax· LMArena data0.0623
23Claude Sonnet 5.5High· LMArena data0.0573
24Hy 4· LMArena data0.056
25GPT-6 LunaMedium· LMArena data0.0554
26GPT-6.1 SolMedium· LMArena data0.0511
27Gemini 3.7 FlashHigh· LMArena data0.051
28GPT-6.1 SolMax· LMArena data0.0434
29GPT-6.2 SolMax· LMArena data0.0428
30Muse WonderMax· LMArena data0.0343
31Qwen4 MaxxHigh· LMArena data0.0335
32Inkling 2 Small· LMArena data0.0223
33Qwen3.7 Max· LMArena data0.0212
34Gemini 3.1 Pro Preview· LMArena data0.0146
35Muse Spark 1.3Max· LMArena data0.0115
36Beam 1.0 501B A23B· LMArena data0.0113
37DeepSeek V4.1 Flash 1017Max· LMArena data0.0108
38MiMo V2.6 Pro· LMArena data0.00454
39Inkling Small· LMArena data0.00315
40GPT-6 LunaLow· LMArena data0.00287
41GPT-5.6 TerraxHigh· LMArena data-0.00305
42Qwen3.7 Plus· LMArena data-0.00984
43Inkling FP4· LMArena data-0.0114
44Muse Spark 1.1· LMArena data-0.0375
45MiniMax M3· LMArena data-0.04
46Gemini 4 HydrogenHigh· LMArena data-0.0455
47Hy4 preview· LMArena data-0.0463
48Qwen3.8 Max· LMArena data-0.0517
49Muse Spark 1.2xHigh· LMArena data-0.0518
50GLM 5.2Max· LMArena data-0.0551

Showing 1–50 of 68 results.

Data credits: LMArena, CC BY 4.0 (licence)