Benchmark
Open Agent Leaderboard AppWorld test_normal
Agentic benchmarks measure how well models carry out tasks that take several steps. Scores are percentages. Higher is better.
- Models measured
- 5
- Scores recorded
- 25
- Sources
- 1
- IQ weight
- 0.92
Score distribution
- 26%
- Mean
- 11%
- Median
- 25.5%
- Std dev
- 0%
- 10th percentile
- 66%
- 90th percentile
- 0%
- Min
This benchmark has a weight of 0.92 for chat models, measured over 5 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.
Leaderboard
| # | Model | Score |
|---|---|---|
| 1 | Claude Opus 4.5agent: smolagents_code, task: appworld_test_normal, subset: test_normal, route: openai/aws· Open Agent Leaderboard results data docs | 70% |
| 2 | Claude Opus 4.5agent: openai_solo, task: appworld_test_normal, subset: test_normal, route: openai/aws· Open Agent Leaderboard results data docs | 68% |
| 3 | Claude Opus 4.5agent: claude_code, task: appworld_test_normal, subset: test_normal, route: openai/aws· Open Agent Leaderboard results data docs | 66% |
| 4 | Claude Opus 4.5agent: tool_calling_with_shortlisting, task: appworld_test_normal, subset: test_normal, route: openai/aws· Open Agent Leaderboard results data docs | 64% |
| 5 | Claude Opus 4.5agent: tool_calling, task: appworld_test_normal, subset: test_normal, route: openai/aws· Open Agent Leaderboard results data docs | 61% |
| 6 | Gemini 3 Pro Previewagent: openai_solo, task: appworld_test_normal, subset: test_normal, route: openai/gcp· Open Agent Leaderboard results data docs | 57% |
| 7 | Gemini 3 Pro Previewagent: tool_calling_with_shortlisting, task: appworld_test_normal, subset: test_normal, route: openai/gcp· Open Agent Leaderboard results data docs | 55% |
| 8 | Gemini 3 Pro Previewagent: tool_calling, task: appworld_test_normal, subset: test_normal, route: openai/gcp· Open Agent Leaderboard results data docs | 50% |
| 9 | Gemini 3 Pro Previewagent: claude_code, task: appworld_test_normal, subset: test_normal, route: openai/gcp· Open Agent Leaderboard results data docs | 36% |
| 10 | GPT 5.2agent: tool_calling_with_shortlisting, task: appworld_test_normal, subset: test_normal, route: openai/azure· Open Agent Leaderboard results data docs | 22% |
| 11 | DeepSeek V3.2agent: smolagents_code, task: appworld_test_normal, subset: test_normal, route: openai/azure· Open Agent Leaderboard results data docs | 13% |
| 12 | Gemini 3 Pro Previewagent: smolagents_code, task: appworld_test_normal, subset: test_normal, route: openai/gcp· Open Agent Leaderboard results data docs | 13% |
| 13 | Kimi K2.5agent: smolagents_code, task: appworld_test_normal, subset: test_normal, route: openai/azure· Open Agent Leaderboard results data docs | 11% |
| 14 | Kimi K2.5agent: tool_calling_with_shortlisting, task: appworld_test_normal, subset: test_normal, route: openai/azure· Open Agent Leaderboard results data docs | 10% |
| 15 | DeepSeek V3.2agent: tool_calling, task: appworld_test_normal, subset: test_normal, route: openai/azure· Open Agent Leaderboard results data docs | 9% |
| 16 | Kimi K2.5agent: tool_calling, task: appworld_test_normal, subset: test_normal, route: openai/azure· Open Agent Leaderboard results data docs | 9% |
| 17 | Kimi K2.5agent: claude_code, task: appworld_test_normal, subset: test_normal, route: openai/azure· Open Agent Leaderboard results data docs | 8% |
| 18 | Kimi K2.5agent: openai_solo, task: appworld_test_normal, subset: test_normal, route: openai/azure· Open Agent Leaderboard results data docs | 8% |
| 19 | GPT 5.2agent: smolagents_code, task: appworld_test_normal, subset: test_normal, route: openai/azure· Open Agent Leaderboard results data docs | 7% |
| 20 | DeepSeek V3.2agent: openai_solo, task: appworld_test_normal, subset: test_normal, route: openai/azure· Open Agent Leaderboard results data docs | 6% |
| 21 | DeepSeek V3.2agent: tool_calling_with_shortlisting, task: appworld_test_normal, subset: test_normal, route: openai/azure· Open Agent Leaderboard results data docs | 4% |
| 22 | DeepSeek V3.2agent: claude_code, task: appworld_test_normal, subset: test_normal, route: openai/azure· Open Agent Leaderboard results data docs | 3% |
| 23 | GPT 5.2agent: claude_code, task: appworld_test_normal, subset: test_normal, route: openai/azure· Open Agent Leaderboard results data docs | 0% |
| 24 | GPT 5.2agent: openai_solo, task: appworld_test_normal, subset: test_normal, route: openai/azure· Open Agent Leaderboard results data docs | 0% |
| 25 | GPT 5.2agent: tool_calling, task: appworld_test_normal, subset: test_normal, route: openai/azure· Open Agent Leaderboard results data docs | 0% |
Data credits: Open Agent Leaderboard results (Exgentic), CDLA-Permissive-2.0; agreement text at https://cdla.dev/permissive-2-0/. Fractions shown as percentages. (licence)