Skip to content

Benchmark

tau2-bench airline

Agentic benchmarks measure how well models carry out tasks that take several steps. Scores are percentages. Higher is better.

Models measured
135
Scores recorded
1083
Sources
1
IQ weight
2.65

Score distribution

68.5%
Mean
73.1%
Median
12.8%
Std dev
47%
10th percentile
77%
90th percentile
0.667%
Min
0.667%81.7%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.65 for chat models, measured over 134 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model
801Nemotron 3.5 Super VLOpenRouter harness, nvidia/nemotron-3.5-super-vl-20261015, PrimeIntellect, public 90-day aggregate· OpenRouter benchmark exports69%
802GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, CoreWeave, public 90-day aggregate· OpenRouter benchmark exports69%
803Qwen3.5 35B A3BOpenRouter harness, qwen/qwen3.5-35b-a3b-20260224, AtlasCloud, public 90-day aggregate· OpenRouter benchmark exports68.9%
804GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, Ambient, public 90-day aggregate· OpenRouter benchmark exports68.8%
805GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, Mistral, public 90-day aggregate· OpenRouter benchmark exports68.8%
806Ling 3.0 FlashOpenRouter harness, inclusionai/ling-3.0-flash-20260723, auto-routing, public 90-day aggregate· OpenRouter benchmark exports68.7%
807GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, Venice, public 90-day aggregate· OpenRouter benchmark exports68.7%
808MiniMax M3.1 FlashOpenRouter harness, minimax/minimax-m3.1-flash-20261019, Minimax, public 90-day aggregate· OpenRouter benchmark exports68.7%
809MiniMax M3.1 FlashOpenRouter harness, minimax/minimax-m3.1-flash-20261019, Wafer, public 90-day aggregate· OpenRouter benchmark exports68.7%
810GPT-6 LunaOpenRouter harness, openai/gpt-6-luna-pro-20260922, Azure, public 90-day aggregate· OpenRouter benchmark exports68.5%
811GPT-6.1 TerraOpenRouter harness, openai/gpt-6.1-terra-20261026, Amazon Bedrock, public 90-day aggregate· OpenRouter benchmark exports68.4%
812GPT-6 LunaOpenRouter harness, openai/gpt-6-luna-20260922, Amazon Bedrock, public 90-day aggregate· OpenRouter benchmark exports68.4%
813Gemma 4 26B A4BOpenRouter harness, google/gemma-4-26b-a4b-it-20260403, PrimeIntellect, public 90-day aggregate· OpenRouter benchmark exports68.3%
814Inkling 2 SmallOpenRouter harness, thinkingmachines/inkling-2-small-20261107, DeepInfra, public 90-day aggregate· OpenRouter benchmark exports68.3%
815GPT-6.1 SolOpenRouter harness, openai/gpt-6.1-sol-20260929, Azure, public 90-day aggregate· OpenRouter benchmark exports68.3%
816GPT-5.6 LunaOpenRouter harness, openai/gpt-5.6-luna-20260709, Amazon Bedrock, public 90-day aggregate· OpenRouter benchmark exports68.2%
817GPT-6.1 SolOpenRouter harness, openai/gpt-6.1-sol-20260929, OpenAI, public 90-day aggregate· OpenRouter benchmark exports68.2%
818Inkling FP4OpenRouter harness, thinkingmachines/inkling-20260715, Together, public 90-day aggregate· OpenRouter benchmark exports68.2%
819Gemma 4 26B A4BOpenRouter harness, google/gemma-4-26b-a4b-it-20260403, NextBit, public 90-day aggregate· OpenRouter benchmark exports68.1%
820Nemotron 3.5 LightningOpenRouter harness, nvidia/nemotron-3.5-lightning-20260807, CoreWeave, public 90-day aggregate· OpenRouter benchmark exports68.1%
821GPT-6 LunaOpenRouter harness, openai/gpt-6-luna-20260922, Azure, public 90-day aggregate· OpenRouter benchmark exports68.1%
822GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, Cloudflare, public 90-day aggregate· OpenRouter benchmark exports68%
823Gemma 4 26B A4BOpenRouter harness, google/gemma-4-26b-a4b-it-20260403, Novita, public 90-day aggregate· OpenRouter benchmark exports68%
824Gemma 4 26B A4BOpenRouter harness, google/gemma-4-26b-a4b-it-20260403, Cloudflare, public 90-day aggregate· OpenRouter benchmark exports68%
825Claude Fable 5.2OpenRouter harness, anthropic/claude-fable-5.2-20261109, Azure, public 90-day aggregate· OpenRouter benchmark exports68%
826Claude Fable 5.2OpenRouter harness, anthropic/claude-fable-5.2-20261109, Google, public 90-day aggregate· OpenRouter benchmark exports68%
827GLM 4.7OpenRouter harness, z-ai/glm-4.7-20251222, StreamLake, public 90-day aggregate· OpenRouter benchmark exports68%
828GPT-6 AstraOpenRouter harness, openai/gpt-6-astra-20260903, auto-routing, public 90-day aggregate· OpenRouter benchmark exports68%
829GPT-6 LunaOpenRouter harness, openai/gpt-6-luna-20260922, OpenAI, public 90-day aggregate· OpenRouter benchmark exports68%
830MiniMax M2.5OpenRouter harness, minimax/minimax-m2.5-20260211, SiliconFlow, public 90-day aggregate· OpenRouter benchmark exports68%
831Nemotron 3.5 LightningOpenRouter harness, nvidia/nemotron-3.5-lightning-20260807, Fireworks, public 90-day aggregate· OpenRouter benchmark exports68%
832Gemma 4 26B A4BOpenRouter harness, google/gemma-4-26b-a4b-it-20260403, DekaLLM, public 90-day aggregate· OpenRouter benchmark exports68%
833Gemma 4 26B A4BOpenRouter harness, google/gemma-4-26b-a4b-it-20260403, auto-routing, public 90-day aggregate· OpenRouter benchmark exports68%
834MiniMax M3.1 FlashOpenRouter harness, minimax/minimax-m3.1-flash-20261019, Tenstorrent, public 90-day aggregate· OpenRouter benchmark exports67.8%
835DeepSeek V4 Flash Vision ExpOpenRouter harness, deepseek/deepseek-v4-flash-vision-exp-20260821, Fireworks, public 90-day aggregate· OpenRouter benchmark exports67.8%
836GPT-6.1 TerraOpenRouter harness, openai/gpt-6.1-terra-20261026, Azure, public 90-day aggregate· OpenRouter benchmark exports67.8%
837Gemma 4 26B A4BOpenRouter harness, google/gemma-4-26b-a4b-it-20260403, Io Net, public 90-day aggregate· OpenRouter benchmark exports67.7%
838MiniMax M2.5OpenRouter harness, minimax/minimax-m2.5-20260211, Minimax, public 90-day aggregate· OpenRouter benchmark exports67.7%
839GPT-6 AstraOpenRouter harness, openai/gpt-6-astra-20260903, OpenAI, public 90-day aggregate· OpenRouter benchmark exports67.7%
840GPT-6 AstraOpenRouter harness, openai/gpt-6-astra-20260903, Amazon Bedrock, public 90-day aggregate· OpenRouter benchmark exports67.6%
841MiniMax M3.1 FlashOpenRouter harness, minimax/minimax-m3.1-flash-20261019, DigitalOcean, public 90-day aggregate· OpenRouter benchmark exports67.6%
842Kimi K2.7 CodeOpenRouter harness, moonshotai/kimi-k2.7-code-20260612, Tenstorrent, public 90-day aggregate· OpenRouter benchmark exports67.5%
843GPT 5OpenRouter harness, openai/gpt-5-2025-08-07, auto-routing, public 90-day aggregate· OpenRouter benchmark exports67.3%
844Inkling FP4OpenRouter harness, thinkingmachines/inkling-20260715, DeepInfra, public 90-day aggregate· OpenRouter benchmark exports67.3%
845GPT-5.6 SolOpenRouter harness, openai/gpt-5.6-sol-pro-20260709, Azure, public 90-day aggregate, pro· OpenRouter benchmark exports67.3%
846DeepSeek V4 Flash 0731OpenRouter harness, deepseek/deepseek-v4-flash-20260731, AkashML, public 90-day aggregate· OpenRouter benchmark exports67.3%
847GPT OSS 120BOpenRouter harness, openai/gpt-oss-120b, Io Net, public 90-day aggregate· OpenRouter benchmark exports67.3%
848GPT-6 LunaOpenRouter harness, openai/gpt-6-luna-20260922, auto-routing, public 90-day aggregate· OpenRouter benchmark exports67.3%
849Kimi K2.5OpenRouter harness, moonshotai/kimi-k2.5-0127, Novita, public 90-day aggregate· OpenRouter benchmark exports67.3%
850MiniMax M2.7OpenRouter harness, minimax/minimax-m2.7-20260318, DeepInfra, public 90-day aggregate· OpenRouter benchmark exports67.3%

Showing 801–850 of 1,083 results.

Tokens the model generated per task, as measured by the source. Comparable within this benchmark only.

Data credits: OpenRouter public benchmark leaderboard and search-lane exports, CC BY 4.0. Fractional source scores converted to percentages; immutable dated snapshots. (licence)