Skip to content

Benchmark

Claw-Eval v1.1 · Overall · Pass@3

Scores are percentages. Higher is better.

Models and systems measured
24
Scores recorded
24
Sources
1
IQ weight
0.78

Score distribution

75%
Mean
76.9%
Median
10.2%
Std dev
69.3%
10th percentile
82.9%
90th percentile
30.7%
Min
30.7%82.9%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.78 for chat models, measured over 19 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model or systemScore
1Claude Sonnet 4.6Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs82.9%
2DeepSeek V4 Pro 0813Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs82.9%
3Gemini 3.1 Pro PreviewClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs82.9%
4Claude Opus 4.6Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs82.4%
5Kimi K2.6Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval, Moonshot AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.80.9%
6MiMo V2.5Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs80.9%
7MiMo V2.5 ProClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs80.9%
8GLM 5.1Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs80.4%
9DeepSeek V4 FlashClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs78.9%
10GPT 5.4Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs78.4%
11Qwen3.6 PlusClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs77.9%
12MiMo V2 ProClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs76.4%
13Kimi K2.5Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs75.4%
14MiniMax M2.7Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs74.4%
15MiMo V2 OmniClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs72.9%
16Agnes 2.0 FlashClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs71.9%
17Qwen3.5 397B A17BClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs71.9%
18Gemini 3 FlashClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs70.4%
19DeepSeek V3.2Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs69.3%

Not ranked

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
GLM 5 TurboClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs75.9%
GLM 5V TurboClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs75.9%
Mach-Mind 4 ProClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs69.3%
Muse Spark (version unspecified)Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs76.9%
Nemotron 3 Super (variant unspecified)Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs30.7%

Data credits: BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links.