Benchmark
SWE-Marathon v1.1
Scores are percentages. Higher is better.
Benchmark sourceScores from BenchBoard docs.
- Models and systems measured
- 7
- Scores recorded
- 7
- Sources
- 1
- IQ weight
- 0.39
Score distribution
- 39.7%
- Mean
- 42.5%
- Median
- 7.5%
- Std dev
- 31.9%
- 10th percentile
- 46%
- 90th percentile
- 30%
- Min
This benchmark has a weight of 0.39 for chat models, measured over 5 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.
Leaderboard
| # | Model or system | Score |
|---|---|---|
| 1 | Claude Opus 5native provider harnesses · 8 trials/task · resolution rate · max effort· Reported by SpaceXAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 50% |
| 2 | Grok 4.7native provider harnesses · 8 trials/task · resolution rate · high effort· Reported by SpaceXAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 46% |
| 3 | GPT-5.6 Solnative provider harnesses · 8 trials/task · resolution rate · max effort· Reported by SpaceXAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 42.5% |
| 4 | GPT-5.6 Terranative provider harnesses · 8 trials/task · resolution rate · max effort· Reported by SpaceXAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 32.5% |
| 5 | Claude Sonnet 5native provider harnesses · 8 trials/task · resolution rate · max effort· Reported by SpaceXAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 30% |
Not ranked
The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.
| Model or system | Score |
|---|---|
| Claude Fable 5 with Opus 4.8 fallbacknative provider harnesses · 8 trials/task · resolution rate · max effort· Reported by SpaceXAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 45% |
| Grok 4.6native provider harnesses · 8 trials/task · resolution rate · high effort· Reported by SpaceXAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 31.9% |