Benchmark
ArtifactsBench
Coding benchmarks compare models on programming tasks. Scores are percentages.
All coding benchmarksScores from ArtifactsBench, LLM Stats data.
- Models measured
- 41
- Scores recorded
- 49
- Sources
- 2
- IQ weight
- 0.86
Score distribution
- 45.8%
- Mean
- 45%
- Median
- 11.1%
- Std dev
- 32.8%
- 10th percentile
- 57.7%
- 90th percentile
- 26%
- Min
This benchmark has a weight of 0.86 for chat models, measured over 41 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.
Leaderboard
| # | Model | Score |
|---|---|---|
| 1 | Ling 3.0 Flash· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 77% |
| 2 | GPT 5Judge-Model-Gemini-2.5-Pro | latest· ArtifactsBench | 72.5% |
| 3 | MiniMax M2Judge-Model-Gemini-2.5-Pro | latest· ArtifactsBench | 66.8% |
| 4 | Claude Opus 4.1Judge-Model-Gemini-2.5-Pro | latest· ArtifactsBench | 59.8% |
| 5 | Ling 1tJudge-Model-Gemini-2.5-Pro | latest· ArtifactsBench | 59.3% |
| 6 | Gemini 2.5 ProJudge-Model-Gemini-2.5-Pro | latest· ArtifactsBench | 57.7% |
| 7 | Claude Sonnet 4Judge-Model-Gemini-2.5-Pro | latest· ArtifactsBench | 57.3% |
| 8 | GPT OSS 120BJudge-Model-Gemini-2.5-Pro | latest· ArtifactsBench | 56.9% |
| 9 | Gemini 2.5 ProJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench | 56.8% |
| 10 | Claude Sonnet 4Judge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench | 55.8% |
| 11 | Qwen 3 235B A22B Thinking 2507Judge-Model-Gemini-2.5-Pro | latest· ArtifactsBench | 55% |
| 12 | OpenAI o3Judge-Model-Gemini-2.5-Pro | latest· ArtifactsBench | 54% |
| 13 | Claude Sonnet 3.7Judge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench | 52.2% |
| 14 | GLM 4.5Judge-Model-Gemini-2.5-Pro | latest· ArtifactsBench | 51.3% |
| 15 | Claude Sonnet 3.7Judge-Model-Gemini-2.5-Pro | latest· ArtifactsBench | 51.3% |
| 16 | Doubao Seed 2.1 Pro· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 51% |
| 17 | Qwen3 235B A22B Instruct 2507Judge-Model-Gemini-2.5-Pro | latest· ArtifactsBench | 50.6% |
| 18 | GLM 4.5 AirJudge-Model-Gemini-2.5-Pro | latest· ArtifactsBench | 48.9% |
| 19 | GPT 4.1Judge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench | 48.2% |
| 20 | DeepSeek R1Judge-Model-Gemini-2.5-Pro | latest· ArtifactsBench | 47.7% |
| 21 | Kimi K2 InstructJudge-Model-Gemini-2.5-Pro | latest· ArtifactsBench | 47.7% |
| 22 | Qwen3 CoderJudge-Model-Gemini-2.5-Pro | latest· ArtifactsBench | 47.2% |
| 23 | Doubao Seed 2.1 Turbo· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 47% |
| 24 | GPT 4.1Judge-Model-Gemini-2.5-Pro | latest· ArtifactsBench | 46% |
| 25 | o3 MiniJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench | 45% |
| 26 | DeepSeek R1Judge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench | 44.6% |
| 27 | DeepSeek V3Judge-Model-Gemini-2.5-Pro | latest· ArtifactsBench | 43.5% |
| 28 | Hunyuan A13B InstructJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench | 43% |
| 29 | Qwen 3 235B A22BJudge-Model-Gemini-2.5-Pro | latest· ArtifactsBench | 41.1% |
| 30 | Hunyuan A13B InstructJudge-Model-Gemini-2.5-Pro | latest· ArtifactsBench | 41% |
| 31 | QwQ 32BJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench | 40.8% |
| 32 | Qwen3 30B A3B InstructJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench | 40.1% |
| 33 | Claude 3.5 SonnetJudge-Model-Gemini-2.5-Pro | latest· ArtifactsBench | 39.9% |
| 34 | Qwen3 14BJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench | 39.8% |
| 35 | o1Judge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench | 38.7% |
| 36 | GPT 4oJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench | 38% |
| 37 | Gemma 3 27B ITJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench | 37.2% |
| 38 | Qwen3 8BJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench | 36.5% |
| 39 | DeepSeek V3Judge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench | 35.7% |
| 40 | Gemma 3 12B ITJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench | 35.5% |
| 41 | Qwen 2.5 Coder 32B InstructJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench | 35.3% |
| 42 | Qwen2.5 72B InstructJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench | 34.5% |
| 43 | GPT 4oJudge-Model-Gemini-2.5-Pro | latest· ArtifactsBench | 33.5% |
| 44 | Qwen3 4BJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench | 32.8% |
| 45 | Qwen2.5 32B InstructJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench | 32.1% |
| 46 | Qwen2.5 VL 72B InstructJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench | 31.7% |
| 47 | Qwen 2.5 14B InstructJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench | 30.5% |
| 48 | Qwen2.5 7B InstructJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench | 27.4% |
| 49 | Qwen2.5 Coder 7B InstructJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench | 26% |
Data credits: ArtifactsBench (Tencent Hunyuan), CC BY 4.0 (licence) · Data by LLM Stats