Benchmark
CyBench Unguided
Coding benchmarks compare models on programming tasks. Scores are percentages.
- Models measured
- 24
- Scores recorded
- 27
- Sources
- 2
- IQ weight
- 2.06
Score distribution
- 32.6%
- Mean
- 22.5%
- Median
- 26.1%
- Std dev
- 7.5%
- 10th percentile
- 82%
- 90th percentile
- 5%
- Min
This benchmark has a weight of 2.06 for chat models, measured over 24 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.
Leaderboard
| # | Model | Score |
|---|---|---|
| 1 | Claude Opus 4.7· CyBench official data | 96% |
| 2 | Claude Opus 4.6· CyBench official dataValues from 2 sources
| 93% |
| 3 | Claude Opus 4.5· CyBench official dataValues from 2 sources
| 82% |
| 4 | Muse Spark· CyBench official data | 65.4% |
| 5 | Claude Sonnet 4.5· CyBench official dataValues from 2 sources
| 60% |
| 6 | Grok 4· CyBench official dataValues from 2 sources
| 43% |
| 7 | Claude Opus 4.1· CyBench official dataValues from 2 sources
| 42% |
| 8 | Grok 4.1· Epoch AI Benchmarking Hub data | 39% |
| 9 | Grok 4.1thinking· CyBench official data | 39% |
| 10 | Claude Opus 4· CyBench official dataValues from 2 sources
| 38% |
| 11 | Claude Sonnet 4· CyBench official dataValues from 2 sources
| 35% |
| 12 | Grok 4 Fast· CyBench official dataValues from 2 sources
| 30% |
| 13 | o3 Mini· CyBench official data | 22.5% |
| 14 | o3 Minimedium· Epoch AI Benchmarking Hub data | 22.5% |
| 15 | Claude Sonnet 3.7· CyBench official dataValues from 2 sources
| 20% |
| 16 | Claude 3.5 Sonnet· CyBench official dataValues from 2 sources
| 17.5% |
| 17 | GPT-4.5 Preview· CyBench official dataValues from 2 sources
| 17.5% |
| 18 | GPT 4o· CyBench official dataValues from 2 sources
| 12.5% |
| 19 | Claude 3 Opus· CyBench official dataValues from 2 sources
| 10% |
| 20 | O1 Mini· CyBench official data | 10% |
| 21 | O1 Minimedium· Epoch AI Benchmarking Hub data | 10% |
| 22 | O1 Preview· CyBench official dataValues from 2 sources
| 10% |
| 23 | Gemini 1.5 Pro· CyBench official data | 7.5% |
| 24 | Llama 3.1 405B Instruct· CyBench official dataValues from 2 sources
| 7.5% |
| 25 | Mixtral 8x22B Instruct· CyBench official data | 7.5% |
| 26 | Mixtral 8x22B Instruct v0.1· Epoch AI Benchmarking Hub data | 7.5% |
| 27 | Llama 3 70B Instruct· Epoch AI Benchmarking Hub data | 5% |
Data credits: CyBench — cite Zhang et al. 2025 · Epoch AI Benchmarking Hub, CC BY 4.0 (licence)