Skip to content

Benchmark

SWE-bench-Live MultiLang TypeScript/JavaScript

Coding benchmarks compare models on programming tasks. Scores are percentages. Higher is better.

Models measured
8
Scores recorded
22
Sources
1
IQ weight
0.87

Score distribution

29.3%
Mean
27.2%
Median
16.3%
Std dev
10.5%
10th percentile
48%
90th percentile
8.6%
Min
8.6%68%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.87 for chat models, measured over 8 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT-5.6 Solscaffold: slingshot-v3.4.0, set: tsjs, instances: 219, submitted: 2026-09-28· SWE-bench-Live leaderboard data68%
2Claude Opus 4.6scaffold: ami-agent, set: tsjs, instances: 204, submitted: 2026-07-10· SWE-bench-Live leaderboard data54.9%
3GPT 5.5scaffold: slingshot-v3.0.0, set: tsjs, instances: 204, submitted: 2026-07-30, effort: medium· SWE-bench-Live leaderboard data48%
4GPT 5.5scaffold: swe-agent, set: tsjs, instances: 204, submitted: 2026-06-05, effort: medium· SWE-bench-Live leaderboard data47.5%
5GPT 5.5scaffold: openhands, set: tsjs, instances: 204, submitted: 2026-06-05, effort: medium· SWE-bench-Live leaderboard data45.1%
6GPT 5.5scaffold: claude-code, set: tsjs, instances: 204, submitted: 2026-06-05, effort: medium· SWE-bench-Live leaderboard data42.6%
7GPT 5.5scaffold: slingshot-v2.6.0, set: tsjs, instances: 204, submitted: 2026-06-29, effort: medium· SWE-bench-Live leaderboard data37.7%
8DeepSeek V4 Proscaffold: openhands, set: tsjs, instances: 204, submitted: 2026-06-05· SWE-bench-Live leaderboard data36.8%
9DeepSeek V4 Proscaffold: swe-agent, set: tsjs, instances: 204, submitted: 2026-06-05· SWE-bench-Live leaderboard data36.8%
10DeepSeek V4 Proscaffold: claude-code, set: tsjs, instances: 204, submitted: 2026-06-05· SWE-bench-Live leaderboard data30.9%
11Claude Sonnet 4.5scaffold: claude-code, set: tsjs, instances: 162, submitted: 2026-01-08· SWE-bench-Live leaderboard data27.2%
12Claude Sonnet 4.5scaffold: slingshot-v2.6.0, set: tsjs, instances: 204, submitted: 2026-06-23· SWE-bench-Live leaderboard data25%
13Claude Sonnet 4.5scaffold: swe-agent, set: tsjs, instances: 162, submitted: 2026-01-08· SWE-bench-Live leaderboard data20.4%
14GPT 5.2scaffold: claude-code, set: tsjs, instances: 162, submitted: 2026-01-08, effort: medium, thinking· SWE-bench-Live leaderboard data20.4%
15Claude Sonnet 4.5scaffold: openhands, set: tsjs, instances: 162, submitted: 2026-01-08· SWE-bench-Live leaderboard data19.1%
16GPT 5.2scaffold: swe-agent, set: tsjs, instances: 162, submitted: 2026-01-08, effort: medium, thinking· SWE-bench-Live leaderboard data16%
17DeepSeek V3.1 Terminusscaffold: claude-code, set: tsjs, instances: 162, submitted: 2026-01-08· SWE-bench-Live leaderboard data15.4%
18GPT 5.2scaffold: openhands, set: tsjs, instances: 162, submitted: 2026-01-08, effort: medium, thinking· SWE-bench-Live leaderboard data13%
19DeepSeek V3.1 Terminusscaffold: openhands, set: tsjs, instances: 162, submitted: 2026-01-08· SWE-bench-Live leaderboard data11.1%
20DeepSeek V3.1 Terminusscaffold: swe-agent, set: tsjs, instances: 162, submitted: 2026-01-08· SWE-bench-Live leaderboard data10.5%
21Gemini 3 Flashscaffold: swe-agent, set: tsjs, instances: 162, submitted: 2026-01-08· SWE-bench-Live leaderboard data9.9%
22Gemini 3 Flashscaffold: openhands, set: tsjs, instances: 162, submitted: 2026-01-08· SWE-bench-Live leaderboard data8.6%

Data credits: SWE-bench-Live (Microsoft and contributors), swe-bench-live.github.io. Resolve rates computed from the published report counts.