Skip to content

Benchmark

Claw-Eval v1.1 · Overall · Pass³

Scores are percentages. Higher is better.

Models and systems measured
24
Scores recorded
24
Sources
1
IQ weight
1.04

Score distribution

54.7%
Mean
57.8%
Median
12.4%
Std dev
45.2%
10th percentile
63.8%
90th percentile
5.5%
Min
5.5%70.4%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.04 for chat models, measured over 19 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model or systemScore
1Claude Opus 4.6Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs70.4%
2Claude Sonnet 4.6Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs67.8%
3MiMo V2.5 ProClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs63.8%
4GLM 5.1Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs62.3%
5Kimi K2.6Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval, Moonshot AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.62.3%
6MiMo V2.5Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs62.3%
7GPT 5.4Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs60.3%
8DeepSeek V4 Pro 0813Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs59.8%
9Qwen3.6 PlusClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs58.8%
10DeepSeek V4 FlashClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs57.8%
11Gemini 3.1 Pro PreviewClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs57.8%
12MiMo V2 ProClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs57.8%
13Qwen3.5 397B A17BClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs56.8%
14Kimi K2.5Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs52.3%
15Agnes 2.0 FlashClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs51.8%
16Gemini 3 FlashClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs49.2%
17MiniMax M2.7Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs48.7%
18MiMo V2 OmniClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs45.2%
19DeepSeek V3.2Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs40.2%

Not ranked

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
GLM 5 TurboClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs55.8%
GLM 5V TurboClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs53.8%
Mach-Mind 4 ProClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs49.2%
Muse Spark (version unspecified)Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs63.8%
Nemotron 3 Super (variant unspecified)Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs5.5%

Data credits: BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links.