Skip to content

Benchmark

CyBench Unguided

Coding benchmarks compare models on programming tasks. Scores are percentages.

Models measured
24
Scores recorded
27
Sources
2
IQ weight
2.06

Score distribution

32.6%
Mean
22.5%
Median
26.1%
Std dev
7.5%
10th percentile
82%
90th percentile
5%
Min
5%96%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.06 for chat models, measured over 24 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Opus 4.7· CyBench official data96%
2Claude Opus 4.6· CyBench official data
Values from 2 sources
93%
3Claude Opus 4.5· CyBench official data
Values from 2 sources
82%
4Muse Spark· CyBench official data65.4%
5Claude Sonnet 4.5· CyBench official data
Values from 2 sources
60%
6Grok 4· CyBench official data
Values from 2 sources
43%
7Claude Opus 4.1· CyBench official data
Values from 2 sources
42%
8Grok 4.1· Epoch AI Benchmarking Hub data39%
9Grok 4.1thinking· CyBench official data39%
10Claude Opus 4· CyBench official data
Values from 2 sources
38%
11Claude Sonnet 4· CyBench official data
Values from 2 sources
35%
12Grok 4 Fast· CyBench official data
Values from 2 sources
30%
13o3 Mini· CyBench official data22.5%
14o3 Minimedium· Epoch AI Benchmarking Hub data22.5%
15Claude Sonnet 3.7· CyBench official data
Values from 2 sources
20%
16Claude 3.5 Sonnet· CyBench official data
Values from 2 sources
17.5%
17GPT-4.5 Preview· CyBench official data
Values from 2 sources
17.5%
18GPT 4o· CyBench official data
Values from 2 sources
12.5%
19Claude 3 Opus· CyBench official data
Values from 2 sources
10%
20O1 Mini· CyBench official data10%
21O1 Minimedium· Epoch AI Benchmarking Hub data10%
22O1 Preview· CyBench official data
Values from 2 sources
10%
23Gemini 1.5 Pro· CyBench official data7.5%
24Llama 3.1 405B Instruct· CyBench official data
Values from 2 sources
7.5%
25Mixtral 8x22B Instruct· CyBench official data7.5%
26Mixtral 8x22B Instruct v0.1· Epoch AI Benchmarking Hub data7.5%
27Llama 3 70B Instruct· Epoch AI Benchmarking Hub data5%

Data credits: CyBench — cite Zhang et al. 2025 · Epoch AI Benchmarking Hub, CC BY 4.0 (licence)