Benchmark
CyberGym
Coding benchmarks compare models on programming tasks. Scores are percentages. Higher is better.
- Models measured
- 26
- Scores recorded
- 48
- Sources
- 2
- IQ weight
- 2.23
Score distribution
- 77%
- Mean
- 80.3%
- Median
- 11.9%
- Std dev
- 66.6%
- 10th percentile
- 86.2%
- 90th percentile
- 41.3%
- Min
This benchmark has a weight of 2.23 for chat models, measured over 26 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.
Leaderboard
| # | Model | Score |
|---|---|---|
| 1 | MiMo V2.6 Flash· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 95.1% |
| 2 | MiMo V2.6 Pro· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 94% |
| 3 | DeepSeek V4.1 Flash· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 88.1% |
| 4 | DeepSeek V4.1 FlashDeepSeek V4.1 launch chart · exact public value · setting not further specified· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 88.1% |
| 5 | DeepSeek V4.1 FlashPass@1· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 88.1% |
| 6 | Gemini 3.8 Flash Cyber· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 86.2% |
| 7 | GLM 5.3· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 84.5% |
| 8 | GLM 5.3Atria Dawn Preview official card / report Table 1 · published comparison run · model snapshots and reasoning tiers not specified unless stated · Claude Code v2.1.119 · high thinking effort · four-hour-ten-minute limit · 2 CPUs / 4 GB· Reported by Shanghai AI Laboratory· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 84.5% |
| 9 | GLM 5.3Claude Code / 无Web工具· Reported by Z.ai· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 84.5% |
| 10 | GLM 5.3DeepSeek V4.1 launch chart · exact public value · setting not further specified· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 84.5% |
| 11 | GPT-5.6 SolDeepSeek V4.1 launch chart · exact public value · setting not further specified· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 84.5% |
| 12 | Claude Fable 5Claude Code / 无Web工具· Reported by Z.ai· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 83.8% |
| 13 | GPT-5.6 SolAtria Dawn Preview official card / report Table 1 · published comparison run · model snapshots and reasoning tiers not specified unless stated · Claude Code v2.1.119 · high thinking effort · four-hour-ten-minute limit · 2 CPUs / 4 GB· Reported by Shanghai AI Laboratory· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 83.6% |
| 14 | GPT-5.6 SolClaude Code / 无Web工具· Reported by Z.ai· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 83.6% |
| 15 | DeepSeek V4 Pro 0813· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 83.3% |
| 16 | DeepSeek V4 Pro 0813Atria Dawn Preview official card / report Table 1 · published comparison run · model snapshots and reasoning tiers not specified unless stated · Claude Code v2.1.119 · high thinking effort · four-hour-ten-minute limit · 2 CPUs / 4 GB· Reported by Shanghai AI Laboratory· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 83.3% |
| 17 | DeepSeek V4 Pro 0813Claude Code / 无Web工具· Reported by Z.ai· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 83.3% |
| 18 | DeepSeek V4 Pro 0813DeepSeek V4.1 launch chart · exact public value · setting not further specified· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 83.3% |
| 19 | MiniMax M3.1 Flash· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 82.2% |
| 20 | MiniMax M3.1 FlashMiniMax M3.1 Flash official-weights model-card run · reasoning effort, sampling, and run count not stated for this benchmark · Claude Code 2.1.263 · 1M context · launch prose rounds to 82%· Reported by MiniMax· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 82.2% |
| 21 | Mistral Large 4· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 82% |
| 22 | GPT 5.5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 81.8% |
| 23 | GPT 5.5xhigh· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 81.8% |
| 24 | Grok 4.7· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 80.3% |
| 25 | Kimi K3Claude Code / 无Web工具· Reported by Z.ai· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 80% |
| 26 | Kimi K3DeepSeek V4.1 launch chart · exact public value · setting not further specified· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 80% |
| 27 | Claude Opus 4.8· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 78.8% |
| 28 | Kimi K3Atria Dawn Preview official card / report Table 1 · published comparison run · model snapshots and reasoning tiers not specified unless stated · Claude Code v2.1.119 · high thinking effort · four-hour-ten-minute limit · 2 CPUs / 4 GB· Reported by Shanghai AI Laboratory· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 78.7% |
| 29 | Qwen3.8 MaxClaude Code / 无Web工具· Reported by Z.ai· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 78.5% |
| 30 | Hy4 preview· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 78.4% |
| 31 | Hy4 previewofficial model-card appendix· Reported by Tencent Hy· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 78.4% |
| 32 | DeepSeek V4 Flash 0731· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 76.7% |
| 33 | DeepSeek V4 Flash 0731DeepSeek V4.1 launch chart · exact public value · setting not further specified· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 76.7% |
| 34 | DeepSeek V4 Flash 0731official model-card setting· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 76.7% |
| 35 | Claude Opus 4.6· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 73.8% |
| 36 | Qwen3.8 MaxAtria Dawn Preview official card / report Table 1 · published comparison run · model snapshots and reasoning tiers not specified unless stated · Claude Code v2.1.119 · high thinking effort · four-hour-ten-minute limit · 2 CPUs / 4 GB· Reported by Shanghai AI Laboratory· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 73.8% |
| 37 | Claude Opus 4.7· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 73.1% |
| 38 | Claude Opus 4.7pass@1 · aggregate suite· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 73.1% |
| 39 | Doubao Seed 2.1 Pro· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 70.2% |
| 40 | GLM 5.1· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 68.7% |
| 41 | GLM 5.1GLM-5.1 official release table· Reported by Z.ai· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 68.7% |
| 42 | Doubao Seed 2.1 Turbo· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 67% |
| 43 | Claude Opus 4.6Anthropic Opus 4.6 launch page · adaptive thinking · max effort unless noted · vulnerability reproduction success rate· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 66.6% |
| 44 | Claude Sonnet 4.6targeted vulnerability reproduction · pass@1 · no thinking · default effort· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 65.2% |
| 45 | Claude Opus 4.5pass@1 · 1,505 tasks · avg@5 · no thinking · 200K context · think tool· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 50.6% |
| 46 | GLM 5GLM-5 official release table· Reported by Z.ai· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 43.2% |
| 47 | Kimi K2.5setting not named by the source· Reported by Moonshot AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 41.3% |
| 48 | Kimi K2.5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 41.3% |
Data credits: BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links. · Data by LLM Stats