Benchmark
Claw-Eval v1.1 · Overall · Pass³
Scores are percentages. Higher is better.
Benchmark sourceScores from BenchBoard docs.
- Models and systems measured
- 24
- Scores recorded
- 24
- Sources
- 1
- IQ weight
- 1.04
Score distribution
- 54.7%
- Mean
- 57.8%
- Median
- 12.4%
- Std dev
- 45.2%
- 10th percentile
- 63.8%
- 90th percentile
- 5.5%
- Min
This benchmark has a weight of 1.04 for chat models, measured over 19 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.
Leaderboard
| # | Model or system | Score |
|---|---|---|
| 1 | Claude Opus 4.6Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs | 70.4% |
| 2 | Claude Sonnet 4.6Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs | 67.8% |
| 3 | MiMo V2.5 ProClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs | 63.8% |
| 4 | GLM 5.1Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs | 62.3% |
| 5 | Kimi K2.6Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval, Moonshot AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 62.3% |
| 6 | MiMo V2.5Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs | 62.3% |
| 7 | GPT 5.4Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs | 60.3% |
| 8 | DeepSeek V4 Pro 0813Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs | 59.8% |
| 9 | Qwen3.6 PlusClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs | 58.8% |
| 10 | DeepSeek V4 FlashClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs | 57.8% |
| 11 | Gemini 3.1 Pro PreviewClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs | 57.8% |
| 12 | MiMo V2 ProClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs | 57.8% |
| 13 | Qwen3.5 397B A17BClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs | 56.8% |
| 14 | Kimi K2.5Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs | 52.3% |
| 15 | Agnes 2.0 FlashClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs | 51.8% |
| 16 | Gemini 3 FlashClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs | 49.2% |
| 17 | MiniMax M2.7Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs | 48.7% |
| 18 | MiMo V2 OmniClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs | 45.2% |
| 19 | DeepSeek V3.2Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs | 40.2% |
Not ranked
The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.
| Model or system | Score |
|---|---|
| GLM 5 TurboClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs | 55.8% |
| GLM 5V TurboClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs | 53.8% |
| Mach-Mind 4 ProClaw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs | 49.2% |
| Muse Spark (version unspecified)Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs | 63.8% |
| Nemotron 3 Super (variant unspecified)Claw-Eval v1.1 · overall · 300 tasks · N=3 · grader: Gemini 3 Flash· Reported by Claw-Eval· via BenchBoard docs | 5.5% |