Skip to content

Benchmark

Protocol Troubleshooting

Higher scores are better.

Models measured
7
Scores recorded
9
Sources
1
IQ weight
0.97

Score distribution

48.3
Mean
58
Median
26.7
Std dev
0.62
10th percentile
68.9
90th percentile
0.518
Min
0.51873.7
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.97 for chat models, measured over 7 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Opus 5.5Opus 5.5 System Card · adaptive thinking · max effort unless stated· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.73.7
2Claude Opus 5Opus 5.5 System Card · adaptive thinking · max effort unless stated· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.68.9
3Claude Sonnet 5.5adaptive thinking · max effort · default sampling · 5-trial mean · context ≤1M unless noted· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.67.3
4GPT-6 AstraOpus 5.5 System Card · adaptive thinking · max effort unless stated· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.66
5Claude Haiku 5.5adaptive thinking · max effort · default sampling · 5-trial mean · context ≤1M unless noted· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.58
6Claude Sonnet 5adaptive thinking · max effort · default sampling · 5-trial mean · context ≤1M unless noted· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.49.9
7Claude Sonnet 5Opus 5.5 System Card · adaptive thinking · max effort unless stated· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.49.9
8Claude Sonnet 5web search allowed· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.0.62
9Claude Opus 4.7bash + web search tools· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.0.518

Data credits: BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links.