Skip to content

Benchmark

RecreationBench paper · Overall · Estimated API cost per task

Lower scores are better.

Models and systems measured
9
Scores recorded
9
Sources
1
IQ weight
No weight published

Score distribution

57.1
Mean
63.1
Median
39.3
Std dev
12.6
10th percentile
115.8
90th percentile
1.2
Min
1.2117.2
The band covers the middle 80% of scores. The line marks the median.

Results

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
Claude Opus 4.8RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · estimated mean API cost per task · 90% cache reads assumed· Reported by QwenLM / RecreationWorld authors· via BenchBoard docs69.2
Claude Opus 5RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · estimated mean API cost per task · 90% cache reads assumed· Reported by QwenLM / RecreationWorld authors· via BenchBoard docs117.2
GLM 5.3RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · estimated mean API cost per task · 90% cache reads assumed· Reported by QwenLM / RecreationWorld authors· via BenchBoard docs71
GPT-5.6 SolRecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · estimated mean API cost per task · 90% cache reads assumed· Reported by QwenLM / RecreationWorld authors· via BenchBoard docs25.5
GPT-6 AstraRecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · estimated mean API cost per task · 90% cache reads assumed· Reported by QwenLM / RecreationWorld authors· via BenchBoard docs115.8
Grok 4.6RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · xhigh reasoning effort per site data · 500k context · 128k max output tokens · estimated mean API cost per task · 90% cache reads assumed· Reported by QwenLM / RecreationWorld authors· via BenchBoard docs12.6
Kimi K3RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · estimated mean API cost per task · 90% cache reads assumed· Reported by QwenLM / RecreationWorld authors· via BenchBoard docs63.1
Qwen3.7 PlusRecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · thinking enabled per site data · 256k context · 32k max output tokens · estimated mean API cost per task · 90% cache reads assumed· Reported by QwenLM / RecreationWorld authors· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.1.2
Qwen3.8 Max 0902RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · xhigh reasoning effort per site data · 256k context · 32k max output tokens · estimated mean API cost per task · 90% cache reads assumed· Reported by QwenLM / RecreationWorld authors· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.38.8

Data credits: BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links.