Skip to content

Benchmark

ProgramBench · Average Pass Rate

Scores are percentages. Higher is better.

Models measured
6
Scores recorded
7
Sources
1
IQ weight
No weight published

Score distribution

84.9%
Mean
85.4%
Median
5.1%
Std dev
79.7%
10th percentile
91.2%
90th percentile
77.3%
Min
77.3%91.2%
The band covers the middle 80% of scores. The line marks the median.

Leaderboard

#ModelScore
1Claude Opus 5.5adaptive thinking · max effort · default sampling · 5-trial mean · context ≤1M unless noted · 166 golden tasks · mini-swe-agent · no 6-hour timeout· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.91.2%
2Claude Opus 5.5Opus 5.5 System Card capability section · 166 golden tasks · mini-swe-agent · no 6-hour timeout· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.91.2%
3Claude Fable 5.1Opus 5.5 System Card capability section · 166 golden tasks · mini-swe-agent · no 6-hour timeout· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.87.6%
4Claude Opus 5Opus 5.5 System Card capability section · 166 golden tasks · mini-swe-agent · no 6-hour timeout· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.85.4%
5Claude Haiku 5.5adaptive thinking · max effort · default sampling · 5-trial mean · context ≤1M unless noted · 166 golden tasks · mini-swe-agent · no 6-hour timeout· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.82%
6Claude Sonnet 5.5adaptive thinking · max effort · default sampling · 5-trial mean · context ≤1M unless noted · 166 golden tasks · mini-swe-agent · no 6-hour timeout· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.79.7%
7Claude Sonnet 5adaptive thinking · max effort · default sampling · 5-trial mean · context ≤1M unless noted · 166 golden tasks · mini-swe-agent · no 6-hour timeout· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.77.3%

Data credits: BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links.