Skip to content

Benchmark

Claw-Eval v1.1 · Multi-turn (38) · Pass³

Scores are percentages. Higher is better.

Models and systems measured
24
Scores recorded
24
Sources
1
IQ weight
2.34

Score distribution

51.9%
Mean
60.5%
Median
18.3%
Std dev
15.8%
10th percentile
65.8%
90th percentile
0%
Min
0%68.4%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.34 for chat models, measured over 19 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model or systemScore
1Claude Opus 4.6Claw-Eval v1.1 · multi-turn · 38 tasks · N=3 · grader + user-agent: Claude Opus 4.6· Reported by Claw-Eval· via BenchBoard docs68.4%
2Claude Sonnet 4.6Claw-Eval v1.1 · multi-turn · 38 tasks · N=3 · grader + user-agent: Claude Opus 4.6· Reported by Claw-Eval· via BenchBoard docs65.8%
3DeepSeek V4 Pro 0813Claw-Eval v1.1 · multi-turn · 38 tasks · N=3 · grader + user-agent: Claude Opus 4.6· Reported by Claw-Eval· via BenchBoard docs65.8%
4Gemini 3.1 Pro PreviewClaw-Eval v1.1 · multi-turn · 38 tasks · N=3 · grader + user-agent: Claude Opus 4.6· Reported by Claw-Eval· via BenchBoard docs65.8%
5Kimi K2.6Claw-Eval v1.1 · multi-turn · 38 tasks · N=3 · grader + user-agent: Claude Opus 4.6· Reported by Claw-Eval· via BenchBoard docs65.8%
6Qwen3.6 PlusClaw-Eval v1.1 · multi-turn · 38 tasks · N=3 · grader + user-agent: Claude Opus 4.6· Reported by Claw-Eval· via BenchBoard docs65.8%
7MiMo V2.5Claw-Eval v1.1 · multi-turn · 38 tasks · N=3 · grader + user-agent: Claude Opus 4.6· Reported by Claw-Eval· via BenchBoard docs63.2%
8MiMo V2.5 ProClaw-Eval v1.1 · multi-turn · 38 tasks · N=3 · grader + user-agent: Claude Opus 4.6· Reported by Claw-Eval· via BenchBoard docs63.2%
9GLM 5.1Claw-Eval v1.1 · multi-turn · 38 tasks · N=3 · grader + user-agent: Claude Opus 4.6· Reported by Claw-Eval· via BenchBoard docs60.5%
10GPT 5.4Claw-Eval v1.1 · multi-turn · 38 tasks · N=3 · grader + user-agent: Claude Opus 4.6· Reported by Claw-Eval· via BenchBoard docs60.5%
11MiMo V2 ProClaw-Eval v1.1 · multi-turn · 38 tasks · N=3 · grader + user-agent: Claude Opus 4.6· Reported by Claw-Eval· via BenchBoard docs60.5%
12DeepSeek V4 FlashClaw-Eval v1.1 · multi-turn · 38 tasks · N=3 · grader + user-agent: Claude Opus 4.6· Reported by Claw-Eval· via BenchBoard docs57.9%
13Gemini 3 FlashClaw-Eval v1.1 · multi-turn · 38 tasks · N=3 · grader + user-agent: Claude Opus 4.6· Reported by Claw-Eval· via BenchBoard docs52.6%
14Qwen3.5 397B A17BClaw-Eval v1.1 · multi-turn · 38 tasks · N=3 · grader + user-agent: Claude Opus 4.6· Reported by Claw-Eval· via BenchBoard docs52.6%
15Kimi K2.5Claw-Eval v1.1 · multi-turn · 38 tasks · N=3 · grader + user-agent: Claude Opus 4.6· Reported by Claw-Eval· via BenchBoard docs50%
16MiniMax M2.7Claw-Eval v1.1 · multi-turn · 38 tasks · N=3 · grader + user-agent: Claude Opus 4.6· Reported by Claw-Eval· via BenchBoard docs44.7%
17DeepSeek V3.2Claw-Eval v1.1 · multi-turn · 38 tasks · N=3 · grader + user-agent: Claude Opus 4.6· Reported by Claw-Eval· via BenchBoard docs31.6%
18MiMo V2 OmniClaw-Eval v1.1 · multi-turn · 38 tasks · N=3 · grader + user-agent: Claude Opus 4.6· Reported by Claw-Eval· via BenchBoard docs15.8%
19Agnes 2.0 FlashClaw-Eval v1.1 · multi-turn · 38 tasks · N=3 · grader + user-agent: Claude Opus 4.6· Reported by Claw-Eval· via BenchBoard docs13.2%

Not ranked

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
GLM 5 TurboClaw-Eval v1.1 · multi-turn · 38 tasks · N=3 · grader + user-agent: Claude Opus 4.6· Reported by Claw-Eval· via BenchBoard docs50%
GLM 5V TurboClaw-Eval v1.1 · multi-turn · 38 tasks · N=3 · grader + user-agent: Claude Opus 4.6· Reported by Claw-Eval· via BenchBoard docs57.9%
Mach-Mind 4 ProClaw-Eval v1.1 · multi-turn · 38 tasks · N=3 · grader + user-agent: Claude Opus 4.6· Reported by Claw-Eval· via BenchBoard docs44.7%
Muse Spark (version unspecified)Claw-Eval v1.1 · multi-turn · 38 tasks · N=3 · grader + user-agent: Claude Opus 4.6· Reported by Claw-Eval· via BenchBoard docs68.4%
Nemotron 3 Super (variant unspecified)Claw-Eval v1.1 · multi-turn · 38 tasks · N=3 · grader + user-agent: Claude Opus 4.6· Reported by Claw-Eval· via BenchBoard docs0%

Data credits: BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links.