Benchmark
CSEBenchmark Programming Skills and Knowledge (PSK)
Scores are percentages. Higher is better.
Benchmark sourceScores from CSEBenchmark data docs.
- Models measured
- 11
- Scores recorded
- 11
- Sources
- 1
- IQ weight
- 1.83
Score distribution
- 78.9%
- Mean
- 84.2%
- Median
- 11.9%
- Std dev
- 69.9%
- 10th percentile
- 89%
- 90th percentile
- 47.8%
- Min
This benchmark has a weight of 1.83 for chat models, measured over 11 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.
Leaderboard
| # | Model | Score |
|---|---|---|
| 1 | DeepSeek V3· CSEBenchmark data docs | 89.9% |
| 2 | GPT 4o· CSEBenchmark data docs | 89% |
| 3 | GPT-4 Turbo· CSEBenchmark data docs | 88.1% |
| 4 | Qwen2.5 72B· CSEBenchmark data docs | 88% |
| 5 | DeepSeek R1· CSEBenchmark data docs | 85.3% |
| 6 | Llama 3.1 70B· CSEBenchmark data docs | 84.2% |
| 7 | Qwen2.5 7B· CSEBenchmark data docs | 77.7% |
| 8 | mixtral-8x7b· CSEBenchmark data docs | 76.3% |
| 9 | GPT-3.5 Turbo· CSEBenchmark data docs | 71.8% |
| 10 | Llama 3.1 8B· CSEBenchmark data docs | 69.9% |
| 11 | Llama 3.2 3B· CSEBenchmark data docs | 47.8% |
Data credits: CSEBenchmark (Wang et al., IEEE S&P 2025). Copyright (c) 2024 NASP. MIT License. (licence)