Skip to content

Benchmark

BenchCAD (with Python tool)

Coding benchmarks compare models on programming tasks. Scores are percentages.

Models measured
8
Scores recorded
8
Sources
1
IQ weight
0.85

Score distribution

88%
Mean
92.7%
Median
8.2%
Std dev
78.2%
10th percentile
96.2%
90th percentile
73.9%
Min
73.9%96.3%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.85 for chat models, measured over 8 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Sonnet 5.5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim96.3%
2Claude Opus 5.5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim96.2%
3GPT-6 Astra· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim95.9%
4Kimi K3.1 Pro· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim92.7%
5Claude Haiku 5.5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim87%
6GPT-5.6 Sol· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim83.4%
7GPT-5.6 Terra· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim78.2%
8GPT-5.6 Luna· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim73.9%

Data credits: Data by LLM Stats