Skip to content

Benchmark

PinchBench v2 · Average Success Rate

Scores are percentages. Higher is better.

Models and systems measured
59
Scores recorded
60
Sources
1
IQ weight
1.10

Score distribution

68.4%
Mean
73.7%
Median
19.6%
Std dev
42.2%
10th percentile
88.9%
90th percentile
3.2%
Min
3.2%93.5%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.10 for chat models, measured over 49 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model or systemScore
1Claude Opus 4.8OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · fast route · average across 5 runs· Reported by PinchBench· via BenchBoard docs93.5%
2Qwen3.7 MaxOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 6 runs· Reported by PinchBench· via BenchBoard docs92.5%
3Claude Opus 4.8OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 5 runs· Reported by PinchBench· via BenchBoard docs90.5%
4Nemotron 3 UltraOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 5 runs· Reported by PinchBench· via BenchBoard docs89.9%
5MiMo V2.5OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 6 runs· Reported by PinchBench· via BenchBoard docs89.7%
6GPT-5.6 LunaOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 5 runs· Reported by PinchBench· via BenchBoard docs88.7%
7Qwen3.6 FlashOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 5 runs· Reported by PinchBench· via BenchBoard docs88.1%
8MiMo V2.5 ProOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 6 runs· Reported by PinchBench· via BenchBoard docs87.5%
9GLM 5.2OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 5 runs· Reported by PinchBench· via BenchBoard docs87%
10Nemotron 3.5 Lightning 30B A3BOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 6 runs· Reported by PinchBench· via BenchBoard docs86.4%
11GPT-5.6 SolOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 5 runs· Reported by PinchBench· via BenchBoard docs84.2%
12DeepSeek V4 FlashOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 7 runs· Reported by PinchBench· via BenchBoard docs81.7%
13Gemini 3.1 Pro PreviewOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 10 runs· Reported by PinchBench· via BenchBoard docs81%
14Gemini 3.1 Flash LiteOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 1 run· Reported by PinchBench· via BenchBoard docs80.5%
15Step 3.5 FlashOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 12 runs· Reported by PinchBench· via BenchBoard docs79.4%
16GPT-5.4 miniOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 17 runs· Reported by PinchBench· via BenchBoard docs79.2%
17Kimi K2.7 CodeOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 5 runs· Reported by PinchBench· via BenchBoard docs76.1%
18Claude Opus 4.7OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 15 runs· Reported by PinchBench· via BenchBoard docs76%
19GPT-5.6 TerraOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 5 runs· Reported by PinchBench· via BenchBoard docs75.9%
20GPT 5.4OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 17 runs· Reported by PinchBench· via BenchBoard docs75.7%
21GPT 5.5OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 16 runs· Reported by PinchBench· via BenchBoard docs75.5%
22Grok 4.5OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 5 runs· Reported by PinchBench· via BenchBoard docs75.2%
23Gemini 3.5 FlashOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 5 runs· Reported by PinchBench· via BenchBoard docs74.2%
24Qwen3.6 PlusOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 20 runs· Reported by PinchBench· via BenchBoard docs72.5%
25Gemini 3 FlashOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 18 runs· Reported by PinchBench· via BenchBoard docs72.1%
26Claude Opus 4.6OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 24 runs· Reported by PinchBench· via BenchBoard docs69.9%
27Fugu UltraOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 5 runs· Reported by PinchBench· via BenchBoard docs69.5%
28Devstral 2 123b Instruct 2512OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 16 runs· Reported by PinchBench· via BenchBoard docs69.4%
29GPT 5.4 NanoOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 17 runs· Reported by PinchBench· via BenchBoard docs69%
30Claude Haiku 4.5OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 17 runs· Reported by PinchBench· via BenchBoard docs67.7%
31MiniMax M2.7OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 13 runs· Reported by PinchBench· via BenchBoard docs66.8%
32Trinity Large ThinkingOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 9 runs· Reported by PinchBench· via BenchBoard docs65.7%
33Claude Sonnet 4.6OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 19 runs· Reported by PinchBench· via BenchBoard docs62.7%
34DeepSeek V4 Pro 0813OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 7 runs· Reported by PinchBench· via BenchBoard docs61.1%
35GLM 5.1OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 6 runs· Reported by PinchBench· via BenchBoard docs59.9%
36Gemma 4 26B A4BOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 17 runs· Reported by PinchBench· via BenchBoard docs56.4%
37Claude Fable 5OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 5 runs· Reported by PinchBench· via BenchBoard docs54.8%
38Kimi K2.5OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 13 runs· Reported by PinchBench· via BenchBoard docs54.6%
39Mistral Large 3 675b InstructOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 17 runs· Reported by PinchBench· via BenchBoard docs54.5%
40Trinity Large PreviewOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 16 runs· Reported by PinchBench· via BenchBoard docs53%
41Gemma 4 31B ITOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 12 runs· Reported by PinchBench· via BenchBoard docs52.7%
42Claude Sonnet 4OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 2 runs· Reported by PinchBench· via BenchBoard docs48.8%
43GPT OSS 120BOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 13 runs· Reported by PinchBench· via BenchBoard docs44.8%
44Nemotron 3 Super 120B A12BOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 15 runs· Reported by PinchBench· via BenchBoard docs42.2%
45Mercury 2OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 20 runs· Reported by PinchBench· via BenchBoard docs39.6%
46Nova 2 LiteOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 17 runs, v1· Reported by PinchBench· via BenchBoard docs37.6%
47GPT OSS 20bOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 10 runs· Reported by PinchBench· via BenchBoard docs36.3%
48GPT-5.5 ProOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 6 runs· Reported by PinchBench· via BenchBoard docs21.4%
49Llama 3.1 70B InstructOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 1 run· Reported by PinchBench· via BenchBoard docs10.7%
50Llama 4 ScoutOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 1 run· Reported by PinchBench· via BenchBoard docs3.2%

Not ranked

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
Aion 3.0OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 5 runs· Reported by PinchBench· via BenchBoard docs61.1%
GLM 5 TurboOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 18 runs· Reported by PinchBench· via BenchBoard docs71.8%
GLM 5V TurboOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 16 runs· Reported by PinchBench· via BenchBoard docs67.6%
Grok 4.20OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 13 runs· Reported by PinchBench· via BenchBoard docs80.3%
Grok 4.3OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 11 runs· Reported by PinchBench· via BenchBoard docs73.7%
Grok Build 0.1OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 5 runs· Reported by PinchBench· via BenchBoard docs88.9%
Ling 2.6 1TOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 1 run· Reported by PinchBench· via BenchBoard docs82.6%
Nemotron 3.5 Bolt 74B-A9B 24GBOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 10 runs· Reported by PinchBench· via BenchBoard docs87.8%
Nemotron 3.5 Bolt 74B-A9B BF16OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 10 runs· Reported by PinchBench· via BenchBoard docs90.4%
Seed-2.0-LiteOpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 25 runs· Reported by PinchBench· via BenchBoard docs75%

Data credits: BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links.