Skip to content

Benchmark

Internal Research Debugging Evaluation

Coding benchmarks compare models on programming tasks. Scores are percentages.

Models measured
5
Scores recorded
5
Sources
1
IQ weight
0.80

Score distribution

68.7%
Mean
68.3%
Median
10.3%
Std dev
50.8%
10th percentile
81.9%
90th percentile
50.8%
Min

This benchmark has a weight of 0.80 for chat models, measured over 5 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT-6.2 Astra· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim81.9%
2GPT-6.2 Sol· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim74.7%
3GPT-5.6 Sol· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim68.3%
4GPT-5.6 Terra· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim67.8%
5GPT-5.6 Luna· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim50.8%

Data credits: Data by LLM Stats