Skip to content

Benchmark

SWE-bench-Live MultiLang C#

Coding benchmarks compare models on programming tasks. Scores are percentages. Higher is better.

Models measured
6
Scores recorded
17
Sources
1
IQ weight
1.09

Score distribution

26.1%
Mean
21.4%
Median
11.6%
Std dev
14.3%
10th percentile
42.3%
90th percentile
14.3%
Min
14.3%44.9%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.09 for chat models, measured over 6 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1DeepSeek V4 Proscaffold: swe-agent, set: csharp, instances: 78, submitted: 2026-06-05· SWE-bench-Live leaderboard data44.9%
2GPT 5.5scaffold: swe-agent, set: csharp, instances: 78, submitted: 2026-06-05, effort: medium· SWE-bench-Live leaderboard data44.9%
3GPT 5.5scaffold: openhands, set: csharp, instances: 78, submitted: 2026-06-05, effort: medium· SWE-bench-Live leaderboard data42.3%
4GPT 5.5scaffold: claude-code, set: csharp, instances: 78, submitted: 2026-06-05, effort: medium· SWE-bench-Live leaderboard data39.7%
5DeepSeek V4 Proscaffold: openhands, set: csharp, instances: 78, submitted: 2026-06-05· SWE-bench-Live leaderboard data38.5%
6DeepSeek V4 Proscaffold: claude-code, set: csharp, instances: 78, submitted: 2026-06-05· SWE-bench-Live leaderboard data37.2%
7Claude Sonnet 4.5scaffold: swe-agent, set: csharp, instances: 28, submitted: 2026-01-08· SWE-bench-Live leaderboard data21.4%
8DeepSeek V3.1 Terminusscaffold: swe-agent, set: csharp, instances: 28, submitted: 2026-01-08· SWE-bench-Live leaderboard data21.4%
9GPT 5.2scaffold: openhands, set: csharp, instances: 28, submitted: 2026-01-08, effort: medium, thinking· SWE-bench-Live leaderboard data21.4%
10GPT 5.2scaffold: swe-agent, set: csharp, instances: 28, submitted: 2026-01-08, effort: medium, thinking· SWE-bench-Live leaderboard data21.4%
11Claude Sonnet 4.5scaffold: openhands, set: csharp, instances: 28, submitted: 2026-01-08· SWE-bench-Live leaderboard data17.9%
12DeepSeek V3.1 Terminusscaffold: openhands, set: csharp, instances: 28, submitted: 2026-01-08· SWE-bench-Live leaderboard data17.9%
13Gemini 3 Flashscaffold: openhands, set: csharp, instances: 28, submitted: 2026-01-08· SWE-bench-Live leaderboard data17.9%
14Claude Sonnet 4.5scaffold: claude-code, set: csharp, instances: 28, submitted: 2026-01-08· SWE-bench-Live leaderboard data14.3%
15DeepSeek V3.1 Terminusscaffold: claude-code, set: csharp, instances: 28, submitted: 2026-01-08· SWE-bench-Live leaderboard data14.3%
16Gemini 3 Flashscaffold: swe-agent, set: csharp, instances: 28, submitted: 2026-01-08· SWE-bench-Live leaderboard data14.3%
17GPT 5.2scaffold: claude-code, set: csharp, instances: 28, submitted: 2026-01-08, effort: medium, thinking· SWE-bench-Live leaderboard data14.3%

Data credits: SWE-bench-Live (Microsoft and contributors), swe-bench-live.github.io. Resolve rates computed from the published report counts.