Skip to content

Benchmark

SWE-bench-Live MultiLang C/C++

Coding benchmarks compare models on programming tasks. Scores are percentages. Higher is better.

Models measured
6
Scores recorded
17
Sources
1
IQ weight
0.39

Score distribution

32.3%
Mean
35.4%
Median
8.7%
Std dev
22.7%
10th percentile
41.7%
90th percentile
10.4%
Min
10.4%43.8%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.39 for chat models, measured over 6 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Sonnet 4.5scaffold: openhands, set: ccpp, instances: 48, submitted: 2026-01-08· SWE-bench-Live leaderboard data43.8%
2Claude Sonnet 4.5scaffold: swe-agent, set: ccpp, instances: 48, submitted: 2026-01-08· SWE-bench-Live leaderboard data43.8%
3GPT 5.2scaffold: swe-agent, set: ccpp, instances: 48, submitted: 2026-01-08, effort: medium, thinking· SWE-bench-Live leaderboard data41.7%
4GPT 5.5scaffold: claude-code, set: ccpp, instances: 66, submitted: 2026-06-05, effort: medium· SWE-bench-Live leaderboard data37.9%
5GPT 5.2scaffold: claude-code, set: ccpp, instances: 48, submitted: 2026-01-08, effort: medium, thinking· SWE-bench-Live leaderboard data37.5%
6GPT 5.2scaffold: openhands, set: ccpp, instances: 48, submitted: 2026-01-08, effort: medium, thinking· SWE-bench-Live leaderboard data37.5%
7Claude Sonnet 4.5scaffold: claude-code, set: ccpp, instances: 48, submitted: 2026-01-08· SWE-bench-Live leaderboard data35.4%
8Gemini 3 Flashscaffold: openhands, set: ccpp, instances: 48, submitted: 2026-01-08· SWE-bench-Live leaderboard data35.4%
9Gemini 3 Flashscaffold: swe-agent, set: ccpp, instances: 48, submitted: 2026-01-08· SWE-bench-Live leaderboard data35.4%
10GPT 5.5scaffold: swe-agent, set: ccpp, instances: 66, submitted: 2026-06-05, effort: medium· SWE-bench-Live leaderboard data34.8%
11GPT 5.5scaffold: openhands, set: ccpp, instances: 66, submitted: 2026-06-05, effort: medium· SWE-bench-Live leaderboard data33.3%
12DeepSeek V3.1 Terminusscaffold: swe-agent, set: ccpp, instances: 48, submitted: 2026-01-08· SWE-bench-Live leaderboard data27.1%
13DeepSeek V3.1 Terminusscaffold: openhands, set: ccpp, instances: 48, submitted: 2026-01-08· SWE-bench-Live leaderboard data25%
14DeepSeek V4 Proscaffold: swe-agent, set: ccpp, instances: 66, submitted: 2026-06-05· SWE-bench-Live leaderboard data24.2%
15DeepSeek V4 Proscaffold: claude-code, set: ccpp, instances: 66, submitted: 2026-06-05· SWE-bench-Live leaderboard data22.7%
16DeepSeek V4 Proscaffold: openhands, set: ccpp, instances: 66, submitted: 2026-06-05· SWE-bench-Live leaderboard data22.7%
17DeepSeek V3.1 Terminusscaffold: claude-code, set: ccpp, instances: 48, submitted: 2026-01-08· SWE-bench-Live leaderboard data10.4%

Data credits: SWE-bench-Live (Microsoft and contributors), swe-bench-live.github.io. Resolve rates computed from the published report counts.