Skip to content

Benchmark

Orak StarCraftII Elo

Agentic benchmarks measure how well models carry out tasks that take several steps. Scores are Elo ratings. Higher ratings are better.

Models measured
4
Scores recorded
4
Sources
1
IQ weight
No weight published

Score distribution

1,518.8
Mean
1,525
Median
20.8
Std dev
1,500
10th percentile
1,550.1
90th percentile
1,500
Min
1,5001,550.1
The band covers the middle 80% of scores. The line marks the median.

Leaderboard

#ModelScore
1Claude Sonnet 3.7zero-shot· Orak leaderboard docs1,550.1
2GPT 4ozero-shot· Orak leaderboard docs1,525
3GPT-4o minizero-shot· Orak leaderboard docs1,500
4o3 Minizero-shot· Orak leaderboard docs1,500

Data credits: Orak leaderboard, KRAFTON AI (Park et al., NeurIPS 2025). Public numeric scores; leaderboard licence not stated (code MIT not applied; CC BY-NC trajectories not used).