Benchmark
TAU2Bench
Scores are percentages. Higher is better.
Benchmark sourceScores from BenchBoard docs.
- Models measured
- 12
- Scores recorded
- 14
- Sources
- 1
- IQ weight
- 1.07
Score distribution
- 78.9%
- Mean
- 80.3%
- Median
- 9.1%
- Std dev
- 61.9%
- 10th percentile
- 89.7%
- 90th percentile
- 61.1%
- Min
This benchmark has a weight of 1.07 for chat models, measured over 5 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.
Leaderboard
| # | Model | Score |
|---|---|---|
| 1 | Gemini 3 FlashRetail + Airline fixed + Telecom average · Sierra framework· Reported by Google DeepMind· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 90.2% |
| 2 | GLM 5GLM-5 official release table· Reported by Z.ai· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 89.7% |
| 3 | Step 3.5 FlashStep 3.5 Flash official headline table · vanilla· Reported by StepFun· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 88.2% |
| 4 | Qwen3.5 397B A17BQwen3.5 release/model card · official setup except airline fixes from Claude Opus 4.5 System Card· Reported by Qwen· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 86.7% |
| 5 | Gemini 3 Pro PreviewGemini 3 Pro Model Card · results as of November 2025· Reported by Google DeepMind· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 85.4% |
| 6 | Qwen3.5 Omni PlusQwen3.5-Omni release · no-thinking setting · airline fixes from Claude Opus 4.5 System Card· Reported by Qwen, Qwen Team· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 81% |
| 7 | DeepSeek V3.2Thinking mode · temperature 1.0 · 128K context · self-play user agent · Airline/Retail/Telecom average· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 80.3% |
| 8 | MiMo V2 FlashMiMo-V2-Flash final evaluation · aggregate · DeepSeek-V3.2 user agent· Reported by Xiaomi MiMo· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 80.3% |
| 9 | Qwen3.5 Omni FlashQwen3.5-Omni release · no-thinking setting · airline fixes from Claude Opus 4.5 System Card· Reported by Qwen, Qwen Team· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 78% |
| 10 | DeepSeek V3.2Non-thinking mode · temperature 1.0 · 128K context · pass@1· Reported by DeepSeek· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 77.2% |
| 11 | Gemma 4 31B ITGemma 4 official model card · aggregate· Reported by Google DeepMind· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 76.9% |
| 12 | Gemma 4 26B A4BGemma 4 official model card · aggregate· Reported by Google DeepMind· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 68.2% |
| 13 | Nemotron 3 Super 120B A12Bofficial blog Figure 3 · rounded · aggregate· Reported by NVIDIA Developer· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 61.9% |
| 14 | Nemotron 3 Super 120B A12BNeMo Evaluator / mostly NeMo Skills · Airline/Retail/Telecom average · default prompt· Reported by NVIDIA· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 61.1% |