Skip to content

Benchmark

Claw-Eval v1.1 · General (161) · Pass@3

Scores are percentages. Higher is better.

Models and systems measured
25
Scores recorded
25
Sources
1
IQ weight
0.19

Score distribution

74.5%
Mean
75.8%
Median
9.1%
Std dev
68.9%
10th percentile
81.4%
90th percentile
34.8%
Min
34.8%83.9%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.19 for chat models, measured over 20 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model or systemScore
1Step 3.7 FlashClaw-Eval v1.1 · general · 161 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs83.9%
2Claude Sonnet 4.6Claw-Eval v1.1 · general · 161 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs81.4%
3DeepSeek V4 Pro 0813Claw-Eval v1.1 · general · 161 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs81.4%
4Claude Opus 4.6Claw-Eval v1.1 · general · 161 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs80.8%
5Gemini 3.1 Pro PreviewClaw-Eval v1.1 · general · 161 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs80.8%
6GLM 5.1Claw-Eval v1.1 · general · 161 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs80.1%
7MiMo V2.5 ProClaw-Eval v1.1 · general · 161 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs79.5%
8Kimi K2.6Claw-Eval v1.1 · general · 161 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs78.9%
9DeepSeek V4 FlashClaw-Eval v1.1 · general · 161 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs78.3%
10MiMo V2.5Claw-Eval v1.1 · general · 161 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs78.3%
11Agnes 2.0 FlashClaw-Eval v1.1 · general · 161 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs77%
12GPT 5.4Claw-Eval v1.1 · general · 161 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs75.8%
13Qwen3.6 PlusClaw-Eval v1.1 · general · 161 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs75.8%
14MiMo V2 OmniClaw-Eval v1.1 · general · 161 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs75.2%
15Kimi K2.5Claw-Eval v1.1 · general · 161 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs73.9%
16MiMo V2 ProClaw-Eval v1.1 · general · 161 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs72.7%
17MiniMax M2.7Claw-Eval v1.1 · general · 161 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs72%
18DeepSeek V3.2Claw-Eval v1.1 · general · 161 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs71.4%
19Qwen3.5 397B A17BClaw-Eval v1.1 · general · 161 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs70.8%
20Gemini 3 FlashClaw-Eval v1.1 · general · 161 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs67.1%

Not ranked

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
GLM 5 TurboClaw-Eval v1.1 · general · 161 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs73.9%
GLM 5V TurboClaw-Eval v1.1 · general · 161 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs73.3%
Mach-Mind 4 ProClaw-Eval v1.1 · general · 161 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs68.9%
Muse Spark (version unspecified)Claw-Eval v1.1 · general · 161 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs75.8%
Nemotron 3 Super (variant unspecified)Claw-Eval v1.1 · general · 161 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs34.8%

Data credits: BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links.