Skip to content

Benchmark

tau2-bench airline

Agentic benchmarks measure how well models carry out tasks that take several steps. Scores are percentages. Higher is better.

Models measured
135
Scores recorded
1083
Sources
1
IQ weight
2.65

Score distribution

68.5%
Mean
73.1%
Median
12.8%
Std dev
47%
10th percentile
77%
90th percentile
0.667%
Min
0.667%81.7%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.65 for chat models, measured over 134 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model
651Ling 3.0 Flash VLOpenRouter harness, inclusionai/ling-3.0-flash-vl-20260910, Novita, public 90-day aggregate· OpenRouter benchmark exports72%
652MiMo V2.6 FlashOpenRouter harness, xiaomi/mimo-v2.6-flash-20260921, Tenstorrent, public 90-day aggregate· OpenRouter benchmark exports72%
653Nemotron 3.5 Super VLOpenRouter harness, nvidia/nemotron-3.5-super-vl-20261015, Venice, public 90-day aggregate· OpenRouter benchmark exports72%
654Qwen3.6 35B A3BOpenRouter harness, qwen/qwen3.6-35b-a3b-20260415, PrimeIntellect, public 90-day aggregate· OpenRouter benchmark exports72%
655Qwen3.7 MaxOpenRouter harness, qwen/qwen3.7-max-20260520, AtlasCloud, public 90-day aggregate· OpenRouter benchmark exports72%
656GLM 4.7OpenRouter harness, z-ai/glm-4.7-20251222, auto-routing, public 90-day aggregate· OpenRouter benchmark exports72%
657GLM 5.1OpenRouter harness, z-ai/glm-5.1-20260406, DigitalOcean, public 90-day aggregate· OpenRouter benchmark exports72%
658DeepSeek V4.1 FlashOpenRouter harness, deepseek/deepseek-v4.1-flash-20260910, Parasail, public 90-day aggregate· OpenRouter benchmark exports72%
659MiniMax M3OpenRouter harness, minimax/minimax-m3-20260531, StreamLake, public 90-day aggregate· OpenRouter benchmark exports71.8%
660DeepSeek V3.2OpenRouter harness, deepseek/deepseek-v3.2-20251201, Phala, public 90-day aggregate· OpenRouter benchmark exports71.8%
661GLM 4.6OpenRouter harness, z-ai/glm-4.6, auto-routing, public 90-day aggregate· OpenRouter benchmark exports71.8%
662GLM 5.1OpenRouter harness, z-ai/glm-5.1-20260406, Phala, public 90-day aggregate· OpenRouter benchmark exports71.8%
663Gemma 4 31B ITOpenRouter harness, google/gemma-4-31b-it-20260402, DekaLLM, public 90-day aggregate· OpenRouter benchmark exports71.7%
664GPT-5 miniOpenRouter harness, openai/gpt-5-mini-2025-08-07, auto-routing, public 90-day aggregate· OpenRouter benchmark exports71.7%
665DeepSeek V4.1 Flash 1017OpenRouter harness, deepseek/deepseek-v4.1-flash-20261017, StreamLake, public 90-day aggregate· OpenRouter benchmark exports71.7%
666DeepSeek V4.1 Flash 1017OpenRouter harness, deepseek/deepseek-v4.1-flash-20261017, Venice, public 90-day aggregate· OpenRouter benchmark exports71.7%
667GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, Relace, public 90-day aggregate· OpenRouter benchmark exports71.7%
668GPT-5.6 SolOpenRouter harness, openai/gpt-5.6-sol-pro-20260709, OpenAI, public 90-day aggregate, pro· OpenRouter benchmark exports71.7%
669MiniMax M3.1 FlashOpenRouter harness, minimax/minimax-m3.1-flash-20261019, Together, public 90-day aggregate· OpenRouter benchmark exports71.6%
670GLM 5.1OpenRouter harness, z-ai/glm-5.1-20260406, SiliconFlow, public 90-day aggregate· OpenRouter benchmark exports71.6%
671Kimi K2.6OpenRouter harness, moonshotai/kimi-k2.6-20260420, Chutes, public 90-day aggregate· OpenRouter benchmark exports71.6%
672Nemotron 3.5 LightningOpenRouter harness, nvidia/nemotron-3.5-lightning-20260807, Wafer, public 90-day aggregate· OpenRouter benchmark exports71.5%
673GLM 5.1OpenRouter harness, z-ai/glm-5.1-20260406, Friendli, public 90-day aggregate· OpenRouter benchmark exports71.5%
674GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, AtlasCloud, public 90-day aggregate· OpenRouter benchmark exports71.5%
675GLM 4.7OpenRouter harness, z-ai/glm-4.7-20251222, Venice, public 90-day aggregate· OpenRouter benchmark exports71.4%
676Kimi K2.6OpenRouter harness, moonshotai/kimi-k2.6-20260420, StreamLake, public 90-day aggregate· OpenRouter benchmark exports71.4%
677MiniMax M3OpenRouter harness, minimax/minimax-m3-20260531, DeepInfra, public 90-day aggregate· OpenRouter benchmark exports71.4%
678GLM 5.3 FlashOpenRouter harness, z-ai/glm-5.3-flash-20260826, GMICloud, public 90-day aggregate· OpenRouter benchmark exports71.4%
679DeepSeek V4 Flash 0731OpenRouter harness, deepseek/deepseek-v4-flash-20260731, Nebius, public 90-day aggregate· OpenRouter benchmark exports71.4%
680MiniMax M2.7OpenRouter harness, minimax/minimax-m2.7-20260318, Novita, public 90-day aggregate· OpenRouter benchmark exports71.3%
681GLM 5OpenRouter harness, z-ai/glm-5-20260211, Novita, public 90-day aggregate· OpenRouter benchmark exports71.3%
682DeepSeek V3.2OpenRouter harness, deepseek/deepseek-v3.2-20251201, Alibaba, public 90-day aggregate· OpenRouter benchmark exports71.3%
683Claude Fable 5.2OpenRouter harness, anthropic/claude-fable-5.2-20261109, Anthropic, public 90-day aggregate· OpenRouter benchmark exports71.3%
684DeepSeek V3.1OpenRouter harness, deepseek/deepseek-chat-v3.1, Novita, public 90-day aggregate· OpenRouter benchmark exports71.3%
685DeepSeek V4.1 FlashOpenRouter harness, deepseek/deepseek-v4.1-flash-20260910, Cloudflare, public 90-day aggregate· OpenRouter benchmark exports71.3%
686Gemini 3.1 Flash LiteOpenRouter harness, google/gemini-3.1-flash-lite-20260507, auto-routing, public 90-day aggregate· OpenRouter benchmark exports71.3%
687GLM 4.7OpenRouter harness, z-ai/glm-4.7-20251222, AtlasCloud, public 90-day aggregate· OpenRouter benchmark exports71.3%
688Kimi K2.6OpenRouter harness, moonshotai/kimi-k2.6-20260420, GMICloud, public 90-day aggregate· OpenRouter benchmark exports71.3%
689Kimi K3OpenRouter harness, moonshotai/kimi-k3-20260715, KT, public 90-day aggregate· OpenRouter benchmark exports71.3%
690Ling 3.0 Flash VLOpenRouter harness, inclusionai/ling-3.0-flash-vl-20260910, DeepInfra, public 90-day aggregate· OpenRouter benchmark exports71.3%
691MiniMax M3.1 FlashOpenRouter harness, minimax/minimax-m3.1-flash-20261019, Fireworks, public 90-day aggregate· OpenRouter benchmark exports71.3%
692Qwen3.8 FlashOpenRouter harness, qwen/qwen3.8-flash-20260826, auto-routing, public 90-day aggregate· OpenRouter benchmark exports71.3%
693GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, Baidu, public 90-day aggregate· OpenRouter benchmark exports71.3%
694Gemini 3.8 FlashOpenRouter harness, google/gemini-3.8-flash-20260902, auto-routing, public 90-day aggregate· OpenRouter benchmark exports71.3%
695GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, Novita, public 90-day aggregate· OpenRouter benchmark exports71.3%
696Kimi K2.7 CodeOpenRouter harness, moonshotai/kimi-k2.7-code-20260612, Novita, public 90-day aggregate· OpenRouter benchmark exports71.3%
697Kimi K2.7 CodeOpenRouter harness, moonshotai/kimi-k2.7-code-20260612, ModelRun, public 90-day aggregate· OpenRouter benchmark exports71.3%
698MiMo V2.5 ProOpenRouter harness, xiaomi/mimo-v2.5-pro-20260422, DeepInfra, public 90-day aggregate· OpenRouter benchmark exports71.3%
699MiniMax M2.7OpenRouter harness, minimax/minimax-m2.7-20260318, GMICloud, public 90-day aggregate· OpenRouter benchmark exports71.3%
700MiniMax M2.7OpenRouter harness, minimax/minimax-m2.7-20260318, Minimax, public 90-day aggregate· OpenRouter benchmark exports71.2%

Showing 651–700 of 1,083 results.

Tokens the model generated per task, as measured by the source. Comparable within this benchmark only.

Data credits: OpenRouter public benchmark leaderboard and search-lane exports, CC BY 4.0. Fractional source scores converted to percentages; immutable dated snapshots. (licence)