Skip to content

Benchmark

AppWorld Test-Challenge Level 2: Scenario Goal Completion

Agentic benchmarks measure how well models carry out tasks that take several steps. Scores are percentages. Higher is better.

Models and systems measured
26
Scores recorded
26
Sources
1
IQ weight
No weight published

Score distribution

23.5%
Mean
14%
Median
26.5%
Std dev
0%
10th percentile
74%
90th percentile
0%
Min
0%80%
The band covers the middle 80% of scores. The line marks the median.

Results

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
Claude Opus 4.6Button Agent (f072cd07-6ae0fbdf)· AppWorld Leaderboard data docs62%
claude-opus-4-7LARA MAS (23715209-d5c9ade3)· AppWorld Leaderboard data docs76%
DeepSeekCoderFullCodeRefl (0465049f-bff9fca2)· AppWorld Leaderboard data docs2%
DeepSeekCoderPlanExec (871bf9fd-4de66f57)· AppWorld Leaderboard data docs0%
DeepSeekCoderReAct (9ae299b8-25c255f5)· AppWorld Leaderboard data docs0%
GLM-5.3AtlasHarness (65df675b-ca5e4771)· AppWorld Leaderboard data docs74%
GLM-5.3-FlashIBM CUGA (a2db3f1e-2c28b1b2)· AppWorld Leaderboard data docs80%
GPT-4 TurboFullCodeRefl (571aa685-a8671c57)· AppWorld Leaderboard data docs4%
GPT-4 TurboIPFunCall (08d2ff3e-a0589430)· AppWorld Leaderboard data docs14%
GPT-4 TurboPlanExec (c722a232-74b9384b)· AppWorld Leaderboard data docs2%
GPT-4 TurboReAct (1f0055c0-123849bb)· AppWorld Leaderboard data docs6%
GPT-4.1IBM CUGA (8a02d49a-0df800d6)· AppWorld Leaderboard data docs42%
gpt-4.1-miniLARA MAS (695a4b55-1b68dde4)· AppWorld Leaderboard data docs18%
GPT-4oFullCodeRefl (d035381e-2d30c0e3)· AppWorld Leaderboard data docs10%
GPT-4oIPFunCall (3a889687-923696fe)· AppWorld Leaderboard data docs16%
GPT-4oPlanExec (ac5b742b-aa0243da)· AppWorld Leaderboard data docs6%
GPT-4oReAct (3414a75f-3df6a61f)· AppWorld Leaderboard data docs12%
GPT-4oReAct + 2 SetBSR Demos (1c12639a-57a0ac34)· AppWorld Leaderboard data docs14%
gpt-5.6-lunakecaipan capybara (1095b768-6513e0c6)· AppWorld Leaderboard data docs48%
gptoss-20bRooAGI_agentbuilder_memtune_rl (5acab4bb-badb978f)· AppWorld Leaderboard data docs2%
LLaMA3-70BFullCodeRefl (baf95530-cee120fc)· AppWorld Leaderboard data docs0%
LLaMA3-70BPlanExec (77f25849-862250c4)· AppWorld Leaderboard data docs0%
LLaMA3-70BReAct (4077db41-07bc5204)· AppWorld Leaderboard data docs0%
Qwen2.5-32BLOOP (e5e08b7e-d8e6474f)· AppWorld Leaderboard data docs22%
Qwen3-14BAlibaba Cloud ApsaraLab AgentRL (cabab065-0caee3c3)· AppWorld Leaderboard data docs40%
Qwen3.5-27BSynthAPI (3bd98253-d4fa6c8e)· AppWorld Leaderboard data docs60%

Data credits: AppWorld leaderboard (Stony Brook NLP). Aggregate scores from the official leaderboard; no results licence asserted. Protected bundles and experiment outputs are not used