Skip to content

Benchmark

ProgramBench

Coding benchmarks compare models on programming tasks. Scores are percentages.

Models measured
41
Scores recorded
41
Sources
2
IQ weight
0.71

Score distribution

54.5%
Mean
55.7%
Median
22.2%
Std dev
20.3%
10th percentile
85.6%
90th percentile
16%
Min
16%91.2%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.71 for chat models, measured over 41 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model
1Claude Opus 5.5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim
Values from 2 sources
91.2%
2Muse Wonder Full· ProgramBench
Values from 2 sources
88.5%
3GPT-6 Astra· ProgramBench86.9%
4Muse Wonder· ProgramBench
Values from 2 sources
86.5%
5Claude Fable 5.2· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim85.6%
6Claude Fable 5.1· ProgramBench82.5%
7Claude Haiku 5.5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim82%
8Claude Sonnet 5.5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim79.7%
9Kimi K3· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim77.8%
10Claude Opus 5· ProgramBench74.7%
11Claude Opus 4.8· ProgramBench70.9%
12Muse Spark 1.3· ProgramBench70.8%
13GPT-5.6 Sol· ProgramBench69.9%
14GPT 5.5· ProgramBench69.5%
15GLM 5.2· ProgramBench
Values from 2 sources
64.6%
16Gemini 3.7 Flash· ProgramBench61.2%
17Muse Spark 1.2· ProgramBench57.2%
18Gemini 3.6 Flash· ProgramBench55.7%
19Claude Opus 4.7· ProgramBench55.1%
20Gemini 3.5 Flash· ProgramBench53.6%
21Kimi K2.7 Code· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim53.6%
22Claude Opus 4.6· ProgramBench52.1%
23Doubao Seed 2.1 Pro· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim50.3%
24Muse Wonder 1.0· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim50%
25Doubao Seed 2.1 Turbo· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim49.4%
26Claude Sonnet 4.6· ProgramBench47.5%
27Muse Spark 1.1· ProgramBench47%
28GPT 5.4· ProgramBench37.7%
29Gemini 3.1 Pro Preview· ProgramBench36.4%
30Gemini 3 Flash· ProgramBench31.8%
31Claude Haiku 4.5· ProgramBench30%
32DeepSeek V4.1 Flash 1017· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim29.1%
33MiMo V2.6 Pro· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim26.5%
34MiMo V2.6 Flash· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim26%
35LongCat 2.5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim25.5%
36DeepSeek V4.1 Flash· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim20.3%
37GLM 5.3· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim19%
38Hy4 preview· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim17.5%
39GPT-5.4 mini· ProgramBench16.4%
40GPT-5 mini· ProgramBench16%
41Hy4 Flash· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim16%

Tokens the model generated per task, as measured by the source. Comparable within this benchmark only.

Data credits: Data by LLM Stats