Skip to content

Benchmark

TAU2Bench

Scores are percentages. Higher is better.

Models measured
12
Scores recorded
14
Sources
1
IQ weight
1.07

Score distribution

78.9%
Mean
80.3%
Median
9.1%
Std dev
61.9%
10th percentile
89.7%
90th percentile
61.1%
Min
61.1%90.2%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.07 for chat models, measured over 5 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemini 3 FlashRetail + Airline fixed + Telecom average · Sierra framework· Reported by Google DeepMind· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.90.2%
2GLM 5GLM-5 official release table· Reported by Z.ai· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.89.7%
3Step 3.5 FlashStep 3.5 Flash official headline table · vanilla· Reported by StepFun· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.88.2%
4Qwen3.5 397B A17BQwen3.5 release/model card · official setup except airline fixes from Claude Opus 4.5 System Card· Reported by Qwen· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.86.7%
5Gemini 3 Pro PreviewGemini 3 Pro Model Card · results as of November 2025· Reported by Google DeepMind· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.85.4%
6Qwen3.5 Omni PlusQwen3.5-Omni release · no-thinking setting · airline fixes from Claude Opus 4.5 System Card· Reported by Qwen, Qwen Team· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.81%
7DeepSeek V3.2Thinking mode · temperature 1.0 · 128K context · self-play user agent · Airline/Retail/Telecom average· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.80.3%
8MiMo V2 FlashMiMo-V2-Flash final evaluation · aggregate · DeepSeek-V3.2 user agent· Reported by Xiaomi MiMo· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.80.3%
9Qwen3.5 Omni FlashQwen3.5-Omni release · no-thinking setting · airline fixes from Claude Opus 4.5 System Card· Reported by Qwen, Qwen Team· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.78%
10DeepSeek V3.2Non-thinking mode · temperature 1.0 · 128K context · pass@1· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.77.2%
11Gemma 4 31B ITGemma 4 official model card · aggregate· Reported by Google DeepMind· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.76.9%
12Gemma 4 26B A4BGemma 4 official model card · aggregate· Reported by Google DeepMind· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.68.2%
13Nemotron 3 Super 120B A12Bofficial blog Figure 3 · rounded · aggregate· Reported by NVIDIA Developer· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.61.9%
14Nemotron 3 Super 120B A12BNeMo Evaluator / mostly NeMo Skills · Airline/Retail/Telecom average · default prompt· Reported by NVIDIA· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.61.1%

Data credits: BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links.