Skip to content

Benchmark

tau2-bench airline

Agentic benchmarks measure how well models carry out tasks that take several steps. Scores are percentages. Higher is better.

Models measured
135
Scores recorded
1083
Sources
1
IQ weight
2.65

Score distribution

68.5%
Mean
73.1%
Median
12.8%
Std dev
47%
10th percentile
77%
90th percentile
0.667%
Min
0.667%81.7%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.65 for chat models, measured over 134 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model
751GPT-6 SolOpenRouter harness, openai/gpt-6-sol-20260922, OpenAI, public 90-day aggregate· OpenRouter benchmark exports70.2%
752GLM 4.7OpenRouter harness, z-ai/glm-4.7-20251222, Z.AI, public 90-day aggregate· OpenRouter benchmark exports70.2%
753Claude Sonnet 4OpenRouter harness, anthropic/claude-4-sonnet-20250522, auto-routing, public 90-day aggregate· OpenRouter benchmark exports70%
754DeepSeek V4 FlashOpenRouter harness, deepseek/deepseek-v4-flash-20260423, Phala, public 90-day aggregate· OpenRouter benchmark exports70%
755DeepSeek V4 Flash 0731OpenRouter harness, deepseek/deepseek-v4-flash-20260731, Ambient, public 90-day aggregate· OpenRouter benchmark exports70%
756DeepSeek V4.1 FlashOpenRouter harness, deepseek/deepseek-v4.1-flash-20260910, Google, public 90-day aggregate· OpenRouter benchmark exports70%
757Gemma 4 31B ITOpenRouter harness, google/gemma-4-31b-it-20260402, SiliconFlow, public 90-day aggregate· OpenRouter benchmark exports70%
758GLM 5.3 FlashOpenRouter harness, z-ai/glm-5.3-flash, digitalocean, public 90-day aggregate· OpenRouter benchmark exports70%
759GPT 5OpenRouter harness, openai/gpt-5-2025-08-07, Azure, public 90-day aggregate· OpenRouter benchmark exports70%
760GPT-6.1 SolOpenRouter harness, openai/gpt-6.1-sol-20260929, auto-routing, public 90-day aggregate· OpenRouter benchmark exports70%
761GPT-6.1 TerraOpenRouter harness, openai/gpt-6.1-terra-20261026, OpenAI, public 90-day aggregate· OpenRouter benchmark exports70%
762Kimi K2.5OpenRouter harness, moonshotai/kimi-k2.5-0127, DeepInfra, public 90-day aggregate· OpenRouter benchmark exports70%
763Kimi K2.5OpenRouter harness, moonshotai/kimi-k2.5-0127, Phala, public 90-day aggregate· OpenRouter benchmark exports70%
764Ling 3.0 Flash VLOpenRouter harness, inclusionai/ling-3.0-flash-vl-20260910, auto-routing, public 90-day aggregate· OpenRouter benchmark exports70%
765MiniMax M2.7OpenRouter harness, minimax/minimax-m2.7-20260318, PrimeIntellect, public 90-day aggregate· OpenRouter benchmark exports70%
766MiniMax M3.1 FlashOpenRouter harness, minimax/minimax-m3.1-flash-20261019, StreamLake, public 90-day aggregate· OpenRouter benchmark exports70%
767GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, StreamLake, public 90-day aggregate· OpenRouter benchmark exports70%
768Qwen3.5 35B A3BOpenRouter harness, qwen/qwen3.5-35b-a3b-20260224, Venice, public 90-day aggregate· OpenRouter benchmark exports69.9%
769GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, DeepInfra, public 90-day aggregate· OpenRouter benchmark exports69.8%
770GLM 5.1OpenRouter harness, z-ai/glm-5.1-20260406, Nebius, public 90-day aggregate· OpenRouter benchmark exports69.8%
771GLM 4.6OpenRouter harness, z-ai/glm-4.6, Venice, public 90-day aggregate· OpenRouter benchmark exports69.8%
772GPT-6 SolOpenRouter harness, openai/gpt-6-sol-20260922, Amazon Bedrock, public 90-day aggregate· OpenRouter benchmark exports69.8%
773MiniMax M3.1 FlashOpenRouter harness, minimax/minimax-m3.1-flash-20261019, Parasail, public 90-day aggregate· OpenRouter benchmark exports69.8%
774GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, BaseTen, public 90-day aggregate· OpenRouter benchmark exports69.7%
775GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, Together, public 90-day aggregate· OpenRouter benchmark exports69.7%
776Qwen3.6 35B A3BOpenRouter harness, qwen/qwen3.6-35b-a3b-20260415, AtlasCloud, public 90-day aggregate· OpenRouter benchmark exports69.7%
777GPT-5.6 LunaOpenRouter harness, openai/gpt-5.6-luna-20260709, auto-routing, public 90-day aggregate· OpenRouter benchmark exports69.7%
778GPT-6 AstraOpenRouter harness, openai/gpt-6-astra-20260903, Azure, public 90-day aggregate· OpenRouter benchmark exports69.7%
779GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, Z.AI, public 90-day aggregate· OpenRouter benchmark exports69.6%
780GLM 5.1OpenRouter harness, z-ai/glm-5.1-20260406, Chutes, public 90-day aggregate· OpenRouter benchmark exports69.6%
781GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, Reka, public 90-day aggregate· OpenRouter benchmark exports69.6%
782Gemma 4 26B A4BOpenRouter harness, google/gemma-4-26b-a4b-it-20260403, DeepInfra, public 90-day aggregate· OpenRouter benchmark exports69.6%
783MiniMax M3.1 FlashOpenRouter harness, minimax/minimax-m3.1-flash-20261019, InferenceNet, public 90-day aggregate· OpenRouter benchmark exports69.6%
784Kimi K2.5OpenRouter harness, moonshotai/kimi-k2.5-0127, auto-routing, public 90-day aggregate· OpenRouter benchmark exports69.6%
785GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, Parasail, public 90-day aggregate· OpenRouter benchmark exports69.5%
786GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, Decart, public 90-day aggregate· OpenRouter benchmark exports69.5%
787GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, Crusoe, public 90-day aggregate· OpenRouter benchmark exports69.3%
788GPT-5.6 LunaOpenRouter harness, openai/gpt-5.6-luna-pro-20260709, Azure, public 90-day aggregate, pro· OpenRouter benchmark exports69.3%
789GPT-6 LunaOpenRouter harness, openai/gpt-6-luna-pro-20260922, OpenAI, public 90-day aggregate· OpenRouter benchmark exports69.3%
790GLM 5OpenRouter harness, z-ai/glm-5-20260211, DigitalOcean, public 90-day aggregate· OpenRouter benchmark exports69.3%
791GPT-6 SolOpenRouter harness, openai/gpt-6-sol-20260922, auto-routing, public 90-day aggregate· OpenRouter benchmark exports69.3%
792Kimi K2.5OpenRouter harness, moonshotai/kimi-k2.5-0127, Moonshot AI, public 90-day aggregate· OpenRouter benchmark exports69.3%
793GLM 4.6OpenRouter harness, z-ai/glm-4.6, DeepInfra, public 90-day aggregate· OpenRouter benchmark exports69.2%
794DeepSeek V3.2OpenRouter harness, deepseek/deepseek-v3.2-20251201, GMICloud, public 90-day aggregate· OpenRouter benchmark exports69.2%
795Nemotron 3.5 Super VLOpenRouter harness, nvidia/nemotron-3.5-super-vl-20261015, BaseTen, public 90-day aggregate· OpenRouter benchmark exports69.1%
796GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, Inceptron, public 90-day aggregate· OpenRouter benchmark exports69.1%
797Gemini 3.1 Flash LiteOpenRouter harness, google/gemini-3.1-flash-lite-20260507, Google AI Studio, public 90-day aggregate· OpenRouter benchmark exports69%
798GLM 5.2OpenRouter harness, z-ai/glm-5.2-20260616, GMICloud, public 90-day aggregate· OpenRouter benchmark exports69%
799GPT-6 AstraOpenRouter harness, openai/gpt-6-astra-pro-20260903, Azure, public 90-day aggregate, pro· OpenRouter benchmark exports69%
800Inkling FP4OpenRouter harness, thinkingmachines/inkling-20260715, BaseTen, public 90-day aggregate· OpenRouter benchmark exports69%

Showing 751–800 of 1,083 results.

Tokens the model generated per task, as measured by the source. Comparable within this benchmark only.

Data credits: OpenRouter public benchmark leaderboard and search-lane exports, CC BY 4.0. Fractional source scores converted to percentages; immutable dated snapshots. (licence)