Benchmark
Open Agent Leaderboard Tau Bench 2 airline
Agentic benchmarks measure how well models carry out tasks that take several steps. Scores are percentages. Higher is better.
- Models measured
- 5
- Scores recorded
- 25
- Sources
- 1
- IQ weight
- 1.04
Score distribution
- 54.5%
- Mean
- 60%
- Median
- 18.9%
- Std dev
- 20%
- 10th percentile
- 70%
- 90th percentile
- 0%
- Min
This benchmark has a weight of 1.04 for chat models, measured over 5 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.
Leaderboard
| # | Model | Score |
|---|---|---|
| 1 | Claude Opus 4.5agent: openai_solo, task: tau2_airline, subset: airline, route: openai/aws· Open Agent Leaderboard results data docs | 74% |
| 2 | Claude Opus 4.5agent: smolagents_code, task: tau2_airline, subset: airline, route: openai/aws· Open Agent Leaderboard results data docs | 72% |
| 3 | Gemini 3 Pro Previewagent: claude_code, task: tau2_airline, subset: airline, route: openai/gcp· Open Agent Leaderboard results data docs | 70% |
| 4 | Gemini 3 Pro Previewagent: tool_calling_with_shortlisting, task: tau2_airline, subset: airline, route: openai/gcp· Open Agent Leaderboard results data docs | 70% |
| 5 | Gemini 3 Pro Previewagent: tool_calling, task: tau2_airline, subset: airline, route: openai/gcp· Open Agent Leaderboard results data docs | 70% |
| 6 | Gemini 3 Pro Previewagent: smolagents_code, task: tau2_airline, subset: airline, route: openai/gcp· Open Agent Leaderboard results data docs | 68% |
| 7 | Claude Opus 4.5agent: claude_code, task: tau2_airline, subset: airline, route: openai/aws· Open Agent Leaderboard results data docs | 66% |
| 8 | Claude Opus 4.5agent: tool_calling_with_shortlisting, task: tau2_airline, subset: airline, route: openai/aws· Open Agent Leaderboard results data docs | 66% |
| 9 | Claude Opus 4.5agent: tool_calling, task: tau2_airline, subset: airline, route: openai/aws· Open Agent Leaderboard results data docs | 66% |
| 10 | Gemini 3 Pro Previewagent: openai_solo, task: tau2_airline, subset: airline, route: openai/gcp· Open Agent Leaderboard results data docs | 62% |
| 11 | Kimi K2.5agent: tool_calling_with_shortlisting, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs | 62% |
| 12 | Kimi K2.5agent: tool_calling, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs | 62% |
| 13 | DeepSeek V3.2agent: smolagents_code, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs | 60% |
| 14 | GPT 5.2agent: smolagents_code, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs | 60% |
| 15 | DeepSeek V3.2agent: tool_calling_with_shortlisting, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs | 56% |
| 16 | DeepSeek V3.2agent: tool_calling, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs | 56% |
| 17 | Kimi K2.5agent: smolagents_code, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs | 56% |
| 18 | GPT 5.2agent: tool_calling_with_shortlisting, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs | 54% |
| 19 | GPT 5.2agent: tool_calling, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs | 54% |
| 20 | GPT 5.2agent: openai_solo, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs | 50% |
| 21 | GPT 5.2agent: claude_code, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs | 48% |
| 22 | DeepSeek V3.2agent: claude_code, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs | 28% |
| 23 | DeepSeek V3.2agent: openai_solo, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs | 20% |
| 24 | Kimi K2.5agent: claude_code, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs | 12% |
| 25 | Kimi K2.5agent: openai_solo, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs | 0% |
Data credits: Open Agent Leaderboard results (Exgentic), CDLA-Permissive-2.0; agreement text at https://cdla.dev/permissive-2-0/. Fractions shown as percentages. (licence)