Skip to content

Benchmark

Open Agent Leaderboard Tau Bench 2 airline

Agentic benchmarks measure how well models carry out tasks that take several steps. Scores are percentages. Higher is better.

Models measured
5
Scores recorded
25
Sources
1
IQ weight
1.04

Score distribution

54.5%
Mean
60%
Median
18.9%
Std dev
20%
10th percentile
70%
90th percentile
0%
Min
0%74%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.04 for chat models, measured over 5 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Opus 4.5agent: openai_solo, task: tau2_airline, subset: airline, route: openai/aws· Open Agent Leaderboard results data docs74%
2Claude Opus 4.5agent: smolagents_code, task: tau2_airline, subset: airline, route: openai/aws· Open Agent Leaderboard results data docs72%
3Gemini 3 Pro Previewagent: claude_code, task: tau2_airline, subset: airline, route: openai/gcp· Open Agent Leaderboard results data docs70%
4Gemini 3 Pro Previewagent: tool_calling_with_shortlisting, task: tau2_airline, subset: airline, route: openai/gcp· Open Agent Leaderboard results data docs70%
5Gemini 3 Pro Previewagent: tool_calling, task: tau2_airline, subset: airline, route: openai/gcp· Open Agent Leaderboard results data docs70%
6Gemini 3 Pro Previewagent: smolagents_code, task: tau2_airline, subset: airline, route: openai/gcp· Open Agent Leaderboard results data docs68%
7Claude Opus 4.5agent: claude_code, task: tau2_airline, subset: airline, route: openai/aws· Open Agent Leaderboard results data docs66%
8Claude Opus 4.5agent: tool_calling_with_shortlisting, task: tau2_airline, subset: airline, route: openai/aws· Open Agent Leaderboard results data docs66%
9Claude Opus 4.5agent: tool_calling, task: tau2_airline, subset: airline, route: openai/aws· Open Agent Leaderboard results data docs66%
10Gemini 3 Pro Previewagent: openai_solo, task: tau2_airline, subset: airline, route: openai/gcp· Open Agent Leaderboard results data docs62%
11Kimi K2.5agent: tool_calling_with_shortlisting, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs62%
12Kimi K2.5agent: tool_calling, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs62%
13DeepSeek V3.2agent: smolagents_code, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs60%
14GPT 5.2agent: smolagents_code, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs60%
15DeepSeek V3.2agent: tool_calling_with_shortlisting, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs56%
16DeepSeek V3.2agent: tool_calling, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs56%
17Kimi K2.5agent: smolagents_code, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs56%
18GPT 5.2agent: tool_calling_with_shortlisting, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs54%
19GPT 5.2agent: tool_calling, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs54%
20GPT 5.2agent: openai_solo, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs50%
21GPT 5.2agent: claude_code, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs48%
22DeepSeek V3.2agent: claude_code, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs28%
23DeepSeek V3.2agent: openai_solo, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs20%
24Kimi K2.5agent: claude_code, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs12%
25Kimi K2.5agent: openai_solo, task: tau2_airline, subset: airline, route: openai/azure· Open Agent Leaderboard results data docs0%

Data credits: Open Agent Leaderboard results (Exgentic), CDLA-Permissive-2.0; agreement text at https://cdla.dev/permissive-2-0/. Fractions shown as percentages. (licence)