Skip to content

Benchmark

ArtifactsBench

Coding benchmarks compare models on programming tasks. Scores are percentages.

Models measured
41
Scores recorded
49
Sources
2
IQ weight
0.86

Score distribution

45.8%
Mean
45%
Median
11.1%
Std dev
32.8%
10th percentile
57.7%
90th percentile
26%
Min
26%77%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.86 for chat models, measured over 41 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Ling 3.0 Flash· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim77%
2GPT 5Judge-Model-Gemini-2.5-Pro | latest· ArtifactsBench72.5%
3MiniMax M2Judge-Model-Gemini-2.5-Pro | latest· ArtifactsBench66.8%
4Claude Opus 4.1Judge-Model-Gemini-2.5-Pro | latest· ArtifactsBench59.8%
5Ling 1tJudge-Model-Gemini-2.5-Pro | latest· ArtifactsBench59.3%
6Gemini 2.5 ProJudge-Model-Gemini-2.5-Pro | latest· ArtifactsBench57.7%
7Claude Sonnet 4Judge-Model-Gemini-2.5-Pro | latest· ArtifactsBench57.3%
8GPT OSS 120BJudge-Model-Gemini-2.5-Pro | latest· ArtifactsBench56.9%
9Gemini 2.5 ProJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench56.8%
10Claude Sonnet 4Judge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench55.8%
11Qwen 3 235B A22B Thinking 2507Judge-Model-Gemini-2.5-Pro | latest· ArtifactsBench55%
12OpenAI o3Judge-Model-Gemini-2.5-Pro | latest· ArtifactsBench54%
13Claude Sonnet 3.7Judge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench52.2%
14GLM 4.5Judge-Model-Gemini-2.5-Pro | latest· ArtifactsBench51.3%
15Claude Sonnet 3.7Judge-Model-Gemini-2.5-Pro | latest· ArtifactsBench51.3%
16Doubao Seed 2.1 Pro· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim51%
17Qwen3 235B A22B Instruct 2507Judge-Model-Gemini-2.5-Pro | latest· ArtifactsBench50.6%
18GLM 4.5 AirJudge-Model-Gemini-2.5-Pro | latest· ArtifactsBench48.9%
19GPT 4.1Judge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench48.2%
20DeepSeek R1Judge-Model-Gemini-2.5-Pro | latest· ArtifactsBench47.7%
21Kimi K2 InstructJudge-Model-Gemini-2.5-Pro | latest· ArtifactsBench47.7%
22Qwen3 CoderJudge-Model-Gemini-2.5-Pro | latest· ArtifactsBench47.2%
23Doubao Seed 2.1 Turbo· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim47%
24GPT 4.1Judge-Model-Gemini-2.5-Pro | latest· ArtifactsBench46%
25o3 MiniJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench45%
26DeepSeek R1Judge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench44.6%
27DeepSeek V3Judge-Model-Gemini-2.5-Pro | latest· ArtifactsBench43.5%
28Hunyuan A13B InstructJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench43%
29Qwen 3 235B A22BJudge-Model-Gemini-2.5-Pro | latest· ArtifactsBench41.1%
30Hunyuan A13B InstructJudge-Model-Gemini-2.5-Pro | latest· ArtifactsBench41%
31QwQ 32BJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench40.8%
32Qwen3 30B A3B InstructJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench40.1%
33Claude 3.5 SonnetJudge-Model-Gemini-2.5-Pro | latest· ArtifactsBench39.9%
34Qwen3 14BJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench39.8%
35o1Judge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench38.7%
36GPT 4oJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench38%
37Gemma 3 27B ITJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench37.2%
38Qwen3 8BJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench36.5%
39DeepSeek V3Judge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench35.7%
40Gemma 3 12B ITJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench35.5%
41Qwen 2.5 Coder 32B InstructJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench35.3%
42Qwen2.5 72B InstructJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench34.5%
43GPT 4oJudge-Model-Gemini-2.5-Pro | latest· ArtifactsBench33.5%
44Qwen3 4BJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench32.8%
45Qwen2.5 32B InstructJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench32.1%
46Qwen2.5 VL 72B InstructJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench31.7%
47Qwen 2.5 14B InstructJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench30.5%
48Qwen2.5 7B InstructJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench27.4%
49Qwen2.5 Coder 7B InstructJudge-Model-Gemini-2.5-Pro-0506 | history· ArtifactsBench26%

Data credits: ArtifactsBench (Tencent Hunyuan), CC BY 4.0 (licence) · Data by LLM Stats