Skip to content

Benchmark

Open Agent Leaderboard Tau Bench 2 telecom

Agentic benchmarks measure how well models carry out tasks that take several steps. Scores are percentages. Higher is better.

Models measured
5
Scores recorded
25
Sources
1
IQ weight
0.44

Score distribution

63.2%
Mean
71%
Median
23.6%
Std dev
18%
10th percentile
84%
90th percentile
0%
Min
0%88%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.44 for chat models, measured over 5 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemini 3 Pro Previewagent: smolagents_code, task: tau2_telecom, subset: telecom, route: openai/gcp· Open Agent Leaderboard results data docs88%
2Claude Opus 4.5agent: openai_solo, task: tau2_telecom, subset: telecom, route: openai/aws· Open Agent Leaderboard results data docs84%
3DeepSeek V3.2agent: smolagents_code, task: tau2_telecom, subset: telecom, route: openai/azure· Open Agent Leaderboard results data docs84%
4Kimi K2.5agent: tool_calling_with_shortlisting, task: tau2_telecom, subset: telecom, route: openai/azure· Open Agent Leaderboard results data docs83%
5Kimi K2.5agent: tool_calling, task: tau2_telecom, subset: telecom, route: openai/azure· Open Agent Leaderboard results data docs83%
6Gemini 3 Pro Previewagent: openai_solo, task: tau2_telecom, subset: telecom, route: openai/gcp· Open Agent Leaderboard results data docs79%
7Claude Opus 4.5agent: claude_code, task: tau2_telecom, subset: telecom, route: openai/aws· Open Agent Leaderboard results data docs76%
8Claude Opus 4.5agent: tool_calling_with_shortlisting, task: tau2_telecom, subset: telecom, route: openai/aws· Open Agent Leaderboard results data docs76%
9Claude Opus 4.5agent: tool_calling, task: tau2_telecom, subset: telecom, route: openai/aws· Open Agent Leaderboard results data docs76%
10Gemini 3 Pro Previewagent: tool_calling_with_shortlisting, task: tau2_telecom, subset: telecom, route: openai/gcp· Open Agent Leaderboard results data docs73%
11Gemini 3 Pro Previewagent: tool_calling, task: tau2_telecom, subset: telecom, route: openai/gcp· Open Agent Leaderboard results data docs73%
12DeepSeek V3.2agent: tool_calling_with_shortlisting, task: tau2_telecom, subset: telecom, route: openai/azure· Open Agent Leaderboard results data docs71%
13DeepSeek V3.2agent: tool_calling, task: tau2_telecom, subset: telecom, route: openai/azure· Open Agent Leaderboard results data docs71%
14Gemini 3 Pro Previewagent: claude_code, task: tau2_telecom, subset: telecom, route: openai/gcp· Open Agent Leaderboard results data docs71%
15GPT 5.2agent: smolagents_code, task: tau2_telecom, subset: telecom, route: openai/azure· Open Agent Leaderboard results data docs71%
16Kimi K2.5agent: smolagents_code, task: tau2_telecom, subset: telecom, route: openai/azure· Open Agent Leaderboard results data docs70.7%
17DeepSeek V3.2agent: claude_code, task: tau2_telecom, subset: telecom, route: openai/azure· Open Agent Leaderboard results data docs61%
18Claude Opus 4.5agent: smolagents_code, task: tau2_telecom, subset: telecom, route: openai/aws· Open Agent Leaderboard results data docs58%
19GPT 5.2agent: claude_code, task: tau2_telecom, subset: telecom, route: openai/azure· Open Agent Leaderboard results data docs55%
20GPT 5.2agent: openai_solo, task: tau2_telecom, subset: telecom, route: openai/azure· Open Agent Leaderboard results data docs53%
21GPT 5.2agent: tool_calling_with_shortlisting, task: tau2_telecom, subset: telecom, route: openai/azure· Open Agent Leaderboard results data docs53%
22GPT 5.2agent: tool_calling, task: tau2_telecom, subset: telecom, route: openai/azure· Open Agent Leaderboard results data docs53%
23DeepSeek V3.2agent: openai_solo, task: tau2_telecom, subset: telecom, route: openai/azure· Open Agent Leaderboard results data docs18%
24Kimi K2.5agent: claude_code, task: tau2_telecom, subset: telecom, route: openai/azure· Open Agent Leaderboard results data docs0%
25Kimi K2.5agent: openai_solo, task: tau2_telecom, subset: telecom, route: openai/azure· Open Agent Leaderboard results data docs0%

Data credits: Open Agent Leaderboard results (Exgentic), CDLA-Permissive-2.0; agreement text at https://cdla.dev/permissive-2-0/. Fractions shown as percentages. (licence)