Skip to content

Benchmark

tau2-bench airline

Agentic benchmarks measure how well models carry out tasks that take several steps. Scores are percentages. Higher is better.

Models measured
135
Scores recorded
1083
Sources
1
IQ weight
2.65

Score distribution

68.5%
Mean
73.1%
Median
12.8%
Std dev
47%
10th percentile
77%
90th percentile
0.667%
Min
0.667%81.7%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.65 for chat models, measured over 134 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model
701MiMo V2.5OpenRouter harness, xiaomi/mimo-v2.5-20260422, DeepInfra, public 90-day aggregate· OpenRouter benchmark exports71.2%
702DeepSeek V4.1 FlashOpenRouter harness, deepseek/deepseek-v4.1-flash-20260910, OpenInference, public 90-day aggregate· OpenRouter benchmark exports71.2%
703DeepSeek V4.1 Flash 1017OpenRouter harness, deepseek/deepseek-v4.1-flash-20261017, River, public 90-day aggregate· OpenRouter benchmark exports71.2%
704DeepSeek V4.1 FlashOpenRouter harness, deepseek/deepseek-v4.1-flash-20260910, Cohere, public 90-day aggregate· OpenRouter benchmark exports71.1%
705DeepSeek V4 FlashOpenRouter harness, deepseek/deepseek-v4-flash-20260423, Novita, public 90-day aggregate· OpenRouter benchmark exports71%
706DeepSeek V4.1 Flash 1017OpenRouter harness, deepseek/deepseek-v4.1-flash-20261017, Cohere, public 90-day aggregate· OpenRouter benchmark exports71%
707GLM 5OpenRouter harness, z-ai/glm-5-20260211, Z.AI, public 90-day aggregate· OpenRouter benchmark exports71%
708GPT-5.6 TerraOpenRouter harness, openai/gpt-5.6-terra-20260709, Amazon Bedrock, public 90-day aggregate· OpenRouter benchmark exports71%
709MiniMax M2.7OpenRouter harness, minimax/minimax-m2.7-20260318, auto-routing, public 90-day aggregate· OpenRouter benchmark exports71%
710MiniMax M3.1 FlashOpenRouter harness, minimax/minimax-m3.1-flash-20261019, ModelRun, public 90-day aggregate· OpenRouter benchmark exports71%
711Qwen 3.8 27BOpenRouter harness, qwen/qwen3.8-27b-20260814, Venice, public 90-day aggregate· OpenRouter benchmark exports71%
712Gemma 4 31B ITOpenRouter harness, google/gemma-4-31b-it-20260402, Novita, public 90-day aggregate· OpenRouter benchmark exports71%
713GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, Friendli, public 90-day aggregate· OpenRouter benchmark exports70.9%
714GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, Alibaba, public 90-day aggregate· OpenRouter benchmark exports70.9%
715MiniMax M3.1 FlashOpenRouter harness, minimax/minimax-m3.1-flash-20261019, Novita, public 90-day aggregate· OpenRouter benchmark exports70.9%
716GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, Nebius, public 90-day aggregate· OpenRouter benchmark exports70.8%
717GPT-5.6 TerraOpenRouter harness, openai/gpt-5.6-terra-20260709, Azure, public 90-day aggregate· OpenRouter benchmark exports70.8%
718Kimi K2.5OpenRouter harness, moonshotai/kimi-k2.5-0127, Venice, public 90-day aggregate· OpenRouter benchmark exports70.8%
719GLM 5.1OpenRouter harness, z-ai/glm-5.1-20260406, auto-routing, public 90-day aggregate· OpenRouter benchmark exports70.8%
720Ling 3.0 Flash FinOpenRouter harness, inclusionai/ling-3.0-flash-fin-20260827, auto-routing, public 90-day aggregate· OpenRouter benchmark exports70.7%
721GLM 5.1OpenRouter harness, z-ai/glm-5.1-20260406, Baidu, public 90-day aggregate· OpenRouter benchmark exports70.7%
722Gemini 3.5 FlashOpenRouter harness, google/gemini-3.5-flash-20260519, Google AI Studio, public 90-day aggregate· OpenRouter benchmark exports70.7%
723GPT-5.6 SolOpenRouter harness, openai/gpt-5.6-sol-pro-20260709, auto-routing, public 90-day aggregate, pro· OpenRouter benchmark exports70.7%
724GPT-6 AstraOpenRouter harness, openai/gpt-6-astra-pro-20260903, OpenAI, public 90-day aggregate, pro· OpenRouter benchmark exports70.7%
725GPT-6.1 SolOpenRouter harness, openai/gpt-6.1-sol-20260929, Amazon Bedrock, public 90-day aggregate· OpenRouter benchmark exports70.7%
726DeepSeek V4.1 Flash 1017OpenRouter harness, deepseek/deepseek-v4.1-flash-20261017, DigitalOcean, public 90-day aggregate· OpenRouter benchmark exports70.7%
727GLM 5.1OpenRouter harness, z-ai/glm-5.1-20260406, Novita, public 90-day aggregate· OpenRouter benchmark exports70.7%
728GPT 5OpenRouter harness, openai/gpt-5-2025-08-07, OpenAI, public 90-day aggregate· OpenRouter benchmark exports70.7%
729GPT-5.6 LunaOpenRouter harness, openai/gpt-5.6-luna-pro-20260709, OpenAI, public 90-day aggregate, pro· OpenRouter benchmark exports70.7%
730MiniMax M3.1 FlashOpenRouter harness, minimax/minimax-m3.1-flash-20261019, SambaNova, public 90-day aggregate· OpenRouter benchmark exports70.6%
731GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, auto-routing, public 90-day aggregate· OpenRouter benchmark exports70.6%
732GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, Wafer, public 90-day aggregate· OpenRouter benchmark exports70.6%
733GLM 5OpenRouter harness, z-ai/glm-5-20260211, Venice, public 90-day aggregate· OpenRouter benchmark exports70.6%
734GPT-5.6 LunaOpenRouter harness, openai/gpt-5.6-luna-20260709, Azure, public 90-day aggregate· OpenRouter benchmark exports70.6%
735GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, Fireworks, public 90-day aggregate· OpenRouter benchmark exports70.6%
736GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, SiliconFlow, public 90-day aggregate· OpenRouter benchmark exports70.5%
737Gemma 4 26B A4BOpenRouter harness, google/gemma-4-26b-a4b-it-20260403, Makora, public 90-day aggregate· OpenRouter benchmark exports70.5%
738DeepSeek V4 Flash 0731OpenRouter harness, deepseek/deepseek-v4-flash-20260731, DigitalOcean, public 90-day aggregate· OpenRouter benchmark exports70.5%
739GPT-5 miniOpenRouter harness, openai/gpt-5-mini-2025-08-07, Azure, public 90-day aggregate· OpenRouter benchmark exports70.5%
740Qwen 3.8 27BOpenRouter harness, qwen/qwen3.8-27b-20260814, Tenstorrent, public 90-day aggregate· OpenRouter benchmark exports70.5%
741GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, Phala, public 90-day aggregate· OpenRouter benchmark exports70.5%
742GLM 4.7OpenRouter harness, z-ai/glm-4.7-20251222, DeepInfra, public 90-day aggregate· OpenRouter benchmark exports70.4%
743Kimi K2.7 CodeOpenRouter harness, moonshotai/kimi-k2.7-code-20260612, CoreWeave, public 90-day aggregate· OpenRouter benchmark exports70.4%
744DeepSeek V4.1 Flash 1017OpenRouter harness, deepseek/deepseek-v4.1-flash-20261017, Makora, public 90-day aggregate· OpenRouter benchmark exports70.4%
745GPT-5.6 LunaOpenRouter harness, openai/gpt-5.6-luna-pro-20260709, auto-routing, public 90-day aggregate, pro· OpenRouter benchmark exports70.3%
746DeepSeek V4 FlashOpenRouter harness, deepseek/deepseek-v4-flash-20260423, Mancer 2, public 90-day aggregate· OpenRouter benchmark exports70.3%
747Qwen3.7 PlusOpenRouter harness, qwen/qwen3.7-plus-20260602, auto-routing, public 90-day aggregate· OpenRouter benchmark exports70.3%
748DeepSeek V4 Pro 0813OpenRouter harness, deepseek/deepseek-v4-pro-20260813, DigitalOcean, public 90-day aggregate· OpenRouter benchmark exports70.3%
749Nemotron 3.5 Super VLOpenRouter harness, nvidia/nemotron-3.5-super-vl-20261015, Io Net, public 90-day aggregate· OpenRouter benchmark exports70.2%
750GPT-5.6 TerraOpenRouter harness, openai/gpt-5.6-terra-20260709, OpenAI, public 90-day aggregate· OpenRouter benchmark exports70.2%

Showing 701–750 of 1,083 results.

Tokens the model generated per task, as measured by the source. Comparable within this benchmark only.

Data credits: OpenRouter public benchmark leaderboard and search-lane exports, CC BY 4.0. Fractional source scores converted to percentages; immutable dated snapshots. (licence)