Skip to content

Benchmark

tau2-bench airline

Agentic benchmarks measure how well models carry out tasks that take several steps. Scores are percentages. Higher is better.

Models measured
135
Scores recorded
1083
Sources
1
IQ weight
2.65

Score distribution

68.5%
Mean
73.1%
Median
12.8%
Std dev
47%
10th percentile
77%
90th percentile
0.667%
Min
0.667%81.7%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.65 for chat models, measured over 134 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model
901MiniMax M2.5OpenRouter harness, minimax/minimax-m2.5-20260211, Venice, public 90-day aggregate· OpenRouter benchmark exports64.7%
902GPT 5.4 NanoOpenRouter harness, openai/gpt-5.4-nano-20260317, auto-routing, public 90-day aggregate· OpenRouter benchmark exports64.7%
903DeepSeek V4.1 FlashOpenRouter harness, deepseek/deepseek-v4.1-flash-20260910, Krea, public 90-day aggregate· OpenRouter benchmark exports64.6%
904GPT OSS 120BOpenRouter harness, openai/gpt-oss-120b, Mancer 2, public 90-day aggregate· OpenRouter benchmark exports64.2%
905GPT 5.4 NanoOpenRouter harness, openai/gpt-5.4-nano-20260317, Azure, public 90-day aggregate· OpenRouter benchmark exports64.2%
906Claude Sonnet 4OpenRouter harness, anthropic/claude-4-sonnet-20250522, Amazon Bedrock, public 90-day aggregate· OpenRouter benchmark exports64%
907GLM 4.7 FlashOpenRouter harness, z-ai/glm-4.7-flash-20260119, Novita, public 90-day aggregate· OpenRouter benchmark exports64%
908Inkling 2 SmallOpenRouter harness, thinkingmachines/inkling-2-small-20261107, Together, public 90-day aggregate· OpenRouter benchmark exports64%
909Ling 3.0 Flash FinOpenRouter harness, inclusionai/ling-3.0-flash-fin-20260827, DeepInfra, public 90-day aggregate· OpenRouter benchmark exports63.8%
910GPT OSS 120BOpenRouter harness, openai/gpt-oss-120b, SambaNova, public 90-day aggregate· OpenRouter benchmark exports63.6%
911GPT OSS 120BOpenRouter harness, openai/gpt-oss-120b, Venice, public 90-day aggregate· OpenRouter benchmark exports63.6%
912Inkling 2 SmallOpenRouter harness, thinkingmachines/inkling-2-small-20261107, Modal, public 90-day aggregate· OpenRouter benchmark exports63.3%
913Inkling 2 SmallOpenRouter harness, thinkingmachines/inkling-2-small-20261107, PrimeIntellect, public 90-day aggregate· OpenRouter benchmark exports63.3%
914GPT OSS 120BOpenRouter harness, openai/gpt-oss-120b, Phala, public 90-day aggregate· OpenRouter benchmark exports63.3%
915GPT OSS 120BOpenRouter harness, openai/gpt-oss-120b, Mara, public 90-day aggregate· OpenRouter benchmark exports63.3%
916Gemini Nano Banana 2.1OpenRouter harness, google/gemini-nano-banana-2.1-20261006, Google, public 90-day aggregate· OpenRouter benchmark exports63.2%
917Inkling 2 SmallOpenRouter harness, thinkingmachines/inkling-2-small-20261107, Cloudflare, public 90-day aggregate· OpenRouter benchmark exports62.7%
918Nemotron 3.5 Super VLOpenRouter harness, nvidia/nemotron-3.5-super-vl-20261015, Crusoe, public 90-day aggregate· OpenRouter benchmark exports62.7%
919Inkling SmallOpenRouter harness, thinkingmachines/inkling-small-20260730, Together, public 90-day aggregate· OpenRouter benchmark exports62.3%
920Kimi K2.6OpenRouter harness, moonshotai/kimi-k2.6-20260420, Decart, public 90-day aggregate· OpenRouter benchmark exports62.3%
921GPT OSS 120BOpenRouter harness, openai/gpt-oss-120b, Cerebras, public 90-day aggregate· OpenRouter benchmark exports62.3%
922GPT OSS 120BOpenRouter harness, openai/gpt-oss-120b, PrimeIntellect, public 90-day aggregate· OpenRouter benchmark exports62%
923Inkling SmallOpenRouter harness, thinkingmachines/inkling-small-20260730, auto-routing, public 90-day aggregate· OpenRouter benchmark exports61.2%
924Kimi K2.5OpenRouter harness, moonshotai/kimi-k2.5-0127, AtlasCloud, public 90-day aggregate· OpenRouter benchmark exports61.2%
925Gemma 4 26B A4BOpenRouter harness, google/gemma-4-26b-a4b-it-20260403, Venice, public 90-day aggregate· OpenRouter benchmark exports60.9%
926GLM 4.6vOpenRouter harness, z-ai/glm-4.6-20251208, auto-routing, public 90-day aggregate· OpenRouter benchmark exports60.7%
927MiniMax M2.5OpenRouter harness, minimax/minimax-m2.5-20260211, Novita, public 90-day aggregate· OpenRouter benchmark exports60.7%
928Inkling SmallOpenRouter harness, thinkingmachines/inkling-small-20260730, DeepInfra, public 90-day aggregate· OpenRouter benchmark exports60.5%
929Qwen3 Coder NextOpenRouter harness, qwen/qwen3-coder-next-2025-02-03, StreamLake, public 90-day aggregate· OpenRouter benchmark exports60.2%
930Gemini 2.5 ProOpenRouter harness, google/gemini-2.5-pro, Google, public 90-day aggregate· OpenRouter benchmark exports60%
931Qwen3 Coder NextOpenRouter harness, qwen/qwen3-coder-next-2025-02-03, auto-routing, public 90-day aggregate· OpenRouter benchmark exports60%
932GLM 4.6vOpenRouter harness, z-ai/glm-4.6-20251208, Z.AI, public 90-day aggregate· OpenRouter benchmark exports59.3%
933Kimi K2.7 CodeOpenRouter harness, moonshotai/kimi-k2.7-code-20260612, BaseTen, public 90-day aggregate· OpenRouter benchmark exports59.3%
934MiniMax M2.5OpenRouter harness, minimax/minimax-m2.5-20260211, AtlasCloud, public 90-day aggregate· OpenRouter benchmark exports59.3%
935DeepSeek V3.2OpenRouter harness, deepseek/deepseek-v3.2-20251201, AtlasCloud, public 90-day aggregate· OpenRouter benchmark exports59.2%
936Gemini 2.5 FlashOpenRouter harness, google/gemini-2.5-flash, Google, public 90-day aggregate· OpenRouter benchmark exports59.1%
937Kimi K2.5OpenRouter harness, moonshotai/kimi-k2.5-0127, Amazon Bedrock, public 90-day aggregate· OpenRouter benchmark exports58.8%
938Gemma 4 31B ITOpenRouter harness, google/gemma-4-31b-it-20260402, Crusoe, public 90-day aggregate· OpenRouter benchmark exports58.8%
939Qwen3 Coder NextOpenRouter harness, qwen/qwen3-coder-next-2025-02-03, Parasail, public 90-day aggregate· OpenRouter benchmark exports58.7%
940DeepSeek R1OpenRouter harness, deepseek/deepseek-r1-0528, Novita, public 90-day aggregate· OpenRouter benchmark exports58.5%
941Gemini 2.5 ProOpenRouter harness, google/gemini-2.5-pro, auto-routing, public 90-day aggregate· OpenRouter benchmark exports58%
942MiniMax M2.5OpenRouter harness, minimax/minimax-m2.5-20260211, StreamLake, public 90-day aggregate· OpenRouter benchmark exports58%
943Gemini 2.5 FlashOpenRouter harness, google/gemini-2.5-flash, auto-routing, public 90-day aggregate· OpenRouter benchmark exports56.7%
944Gemini 2.5 ProOpenRouter harness, google/gemini-2.5-pro, Google AI Studio, public 90-day aggregate· OpenRouter benchmark exports55%
945Gemini 2.5 FlashOpenRouter harness, google/gemini-2.5-flash, Google AI Studio, public 90-day aggregate· OpenRouter benchmark exports54.7%
946GPT OSS 120BOpenRouter harness, openai/gpt-oss-120b, Novita, public 90-day aggregate· OpenRouter benchmark exports54%
947Qwen3 Coder NextOpenRouter harness, qwen/qwen3-coder-next-2025-02-03, Alibaba, public 90-day aggregate· OpenRouter benchmark exports53.7%
948Gemini 2.5 Flash-LiteOpenRouter harness, google/gemini-2.5-flash-lite, Google AI Studio, public 90-day aggregate· OpenRouter benchmark exports53.7%
949Gemini 2.5 Flash-LiteOpenRouter harness, google/gemini-2.5-flash-lite, auto-routing, public 90-day aggregate· OpenRouter benchmark exports53.4%
950GPT OSS 20bOpenRouter harness, openai/gpt-oss-20b, auto-routing, public 90-day aggregate· OpenRouter benchmark exports53.2%

Showing 901–950 of 1,083 results.

Tokens the model generated per task, as measured by the source. Comparable within this benchmark only.

Data credits: OpenRouter public benchmark leaderboard and search-lane exports, CC BY 4.0. Fractional source scores converted to percentages; immutable dated snapshots. (licence)