Skip to content

Benchmark

PaperBench

Coding benchmarks compare models on programming tasks. Scores are percentages. Higher is better.

Models and systems measured
9
Scores recorded
12
Sources
2
IQ weight
1.09

Score distribution

76.5%
Mean
82.1%
Median
15.3%
Std dev
52.6%
10th percentile
93%
90th percentile
52.6%
Min
52.6%93%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.09 for chat models, measured over 8 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model or systemScore
1Qwen3.8 Max· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim93%
2Qwen3.8 MaxQwen表:BasicAgent · Code-Dev · Claude Opus 4.6 judge · 3次均值 · 最长12h· Reported by Qwen· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.93%
3GPT-5.6 SolQwen表:BasicAgent · Code-Dev · Claude Opus 4.6 judge · 3次均值 · 最长12h· Reported by Qwen· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.90.5%
4Claude Fable 5Qwen表:BasicAgent · Code-Dev · Claude Opus 4.6 judge · 3次均值 · 最长12h· Reported by Qwen· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.88.8%
5LongCat 2.5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim82.1%
6Claude Opus 4.8Qwen表:BasicAgent · Code-Dev · Claude Opus 4.6 judge · 3次均值 · 最长12h· Reported by Qwen· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.80.3%
7Qwen3.7 MaxQwen表:BasicAgent · Code-Dev · Claude Opus 4.6 judge · 3次均值 · 最长12h· Reported by Qwen· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.64.8%
8Kimi K2.5setting not named by the source· Reported by Moonshot AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.63.5%
9Kimi K2.5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim63.5%
10MiniMax M3· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim52.6%
11MiniMax M3Claude Code · Ralph-Loop · 12h · 19 papers · Opus 4.6 grader· Reported by MiniMax Research· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.52.6%

Not ranked

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
Qwen3.8-2.4T-A95Blifecycle-page core-benchmark claim· Reported by Alibaba Cloud Model Studio· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.93%

Data credits: BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links. · Data by LLM Stats