Skip to content

Benchmark

tau3-bench v1 voice retail pass^1

Agentic benchmarks measure how well models carry out tasks that take several steps. Scores are percentages. Higher is better.

Models measured
7
Scores recorded
9
Sources
1
IQ weight
No weight published

Score distribution

46.5%
Mean
45.6%
Median
14.8%
Std dev
29.8%
10th percentile
59.6%
90th percentile
26.3%
Min
26.3%78.9%
The band covers the middle 80% of scores. The line marks the median.

Leaderboard

#ModelScore
1GPT-Live 1submission-gpt-live-1-diamond-alpha_openai_2026-09-09, usersim-v1.0, effort-backend-gpt-6-astra-medium· Sierra tau-bench Leaderboard data docs78.9%
2Grok Voice Think Fast 2.0submission-grok-voice-think-fast-2-0_xai_2026-08-06, usersim-v1.0, effort-high· Sierra tau-bench Leaderboard data docs59.6%
3GPT Realtime 2submission-gpt-realtime-2_openai_2026-06-29, usersim-v1.0, effort-xhigh· Sierra tau-bench Leaderboard data docs47.4%
4Qwen3.5 Omni Plus Realtimesubmission-qwen3-5-omni-plus-realtime_qwen_2026-08-06, usersim-v1.0· Sierra tau-bench Leaderboard data docs46.5%
5Gemini 3.1 Flash Live Previewsubmission-gemini-3-1-flash-live-preview-thinking-high_google_2026-04-02, usersim-v1.0, effort-high, thinking, high· Sierra tau-bench Leaderboard data docs45.6%
6GPT Realtime 1.5submission-gpt-realtime-1.5_sierra_2026-03-03, usersim-v1.0· Sierra tau-bench Leaderboard data docs44.7%
7GPT Realtime 2submission-gpt-realtime-2-minimal_openai_2026-07-01, usersim-v1.0, effort-minimal· Sierra tau-bench Leaderboard data docs39.5%
8gemini-live-2.5-flash-native-audiosubmission-gemini-live-2.5-flash_sierra_2026-03-03, usersim-v1.0· Sierra tau-bench Leaderboard data docs29.8%
9Gemini 3.1 Flash Live Previewsubmission-gemini-3-1-flash-live-preview-thinking-minimal_google_2026-04-13, usersim-v1.0, effort-minimal, thinking, minimal· Sierra tau-bench Leaderboard data docs26.3%

Data credits: Sierra Research tau2-bench leaderboard submissions (github.com/sierra-research/tau2-bench), MIT (licence)