Skip to content

Benchmark

SWE-Bench Pro · Average Output Tokens

Agentic benchmarks measure how well models carry out tasks that take several steps. Lower scores are better.

Models measured
2
Scores recorded
2
Sources
1
IQ weight
No weight published

Score distribution

41,487
Mean
67,020
Median
25,533
Std dev
15,954
10th percentile
67,020
90th percentile
15,954
Min
15,95467,020
The band covers the middle 80% of scores. The line marks the median.

Results

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

ModelScore
Claude Opus 4.8max· Reported by SpaceXAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.67,020
Grok 4.5average output tokens per task· Reported by SpaceXAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.15,954

Data credits: BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links.