Skip to content

Benchmark

Multi-Agent Step Race (Lech Mazur)

Agentic benchmarks measure how well models carry out tasks that take several steps. Scores are percentages.

Models measured
52
Scores recorded
59
Sources
1
IQ weight
1.86

Score distribution

2.3%
Mean
2%
Median
1.3%
Std dev
1%
10th percentile
3.8%
90th percentile
0.74%
Min
0.74%5.5%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.86 for chat models, measured over 52 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT 5medium reasoning· Lech Mazur suite data5.5%
2OpenAI o3medium reasoning· Lech Mazur suite data5.3%
3GPT 5.1medium reasoning· Lech Mazur suite data5.3%
4Gemini 3 Pro Preview· Lech Mazur suite data5%
5o1medium reasoning· Lech Mazur suite data4.1%
6Gemini 2.5 Flash· Lech Mazur suite data4%
7Gemini 2.5 Pro· Lech Mazur suite data3.8%
8Grok 4.1 Fastreasoning· Lech Mazur suite data3.8%
9Gemini 2.5 Pro· Lech Mazur suite data3.7%
10DeepSeek V3.2 Exp· Lech Mazur suite data3.7%
11o3 Minihigh reasoning· Lech Mazur suite data3.6%
12Grok 4· Lech Mazur suite data3.5%
13o3 Minimedium reasoning· Lech Mazur suite data3.4%
14Claude Sonnet 4.5thinking, 16k· Lech Mazur suite data3.4%
15Kimi K2 Thinking64k· Lech Mazur suite data3.3%
16Grok 3 MiniHigh, beta· Lech Mazur suite data3.2%
17QwQ 32B16k· Lech Mazur suite data3.2%
18Claude Opus 4.5no reasoning· Lech Mazur suite data3.2%
19DeepSeek R1· Lech Mazur suite data3.2%
20Claude Opus 4.1thinking, 16k· Lech Mazur suite data3.1%
21Claude Sonnet 4thinking, 16k· Lech Mazur suite data3.1%
22Qwen 3 235B A22B· Lech Mazur suite data3.1%
23GPT-5 minimedium reasoning· Lech Mazur suite data2.8%
24Claude Sonnet 3.7thinking, 16k· Lech Mazur suite data2.8%
25o4 Minimedium reasoning· Lech Mazur suite data2.6%
26O1 Mini· Lech Mazur suite data2.4%
27Claude Opus 4thinking, 16k· Lech Mazur suite data2.4%
28Gemini 2.0 Flash Experimentalthink· Lech Mazur suite data2.3%
29GLM 4.6· Lech Mazur suite data2.2%
30Qwen3 30B A3B· Lech Mazur suite data2.2%
31Claude Sonnet 4no reasoning· Lech Mazur suite data2%
32Grok 4.1 Fastnon-reasoning· Lech Mazur suite data1.8%
33GLM 4.5· Lech Mazur suite data1.7%
34Claude 3.5 Sonnet· Lech Mazur suite data1.7%
35GPT OSS 120B· Lech Mazur suite data1.6%
36Claude Opus 4.1no reasoning· Lech Mazur suite data1.6%
37GPT 4o· Lech Mazur suite data1.6%
38Qwen3 Max· Lech Mazur suite data1.5%
39Claude Opus 4no reasoning· Lech Mazur suite data1.5%
40gemini-2.0-pro-exp-02-05· Lech Mazur suite data1.4%
41Qwen2.5-Max· Lech Mazur suite data1.4%
42Mistral Medium 3.1· Lech Mazur suite data1.4%
43Mistral Large 3· Lech Mazur suite data1.4%
44Llama 3.3 70B Instruct· Lech Mazur suite data1.3%
45Grok 3No reasoning, beta· Lech Mazur suite data1.3%
46Qwen2.5 72B· Lech Mazur suite data1.2%
47Claude Sonnet 3.7· Lech Mazur suite data1.2%
48DeepSeek V3· Lech Mazur suite data1.1%
49Llama 4 Maverick· Lech Mazur suite data1.1%
50Kimi K2· Lech Mazur suite data1.1%

Showing 1–50 of 59 results.