Benchmark
tau3-bench v1 voice retail pass^1
Agentic benchmarks measure how well models carry out tasks that take several steps. Scores are percentages. Higher is better.
- Models measured
- 7
- Scores recorded
- 9
- Sources
- 1
- IQ weight
- No weight published
Score distribution
- 46.5%
- Mean
- 45.6%
- Median
- 14.8%
- Std dev
- 29.8%
- 10th percentile
- 59.6%
- 90th percentile
- 26.3%
- Min
Leaderboard
| # | Model | Score |
|---|---|---|
| 1 | GPT-Live 1submission-gpt-live-1-diamond-alpha_openai_2026-09-09, usersim-v1.0, effort-backend-gpt-6-astra-medium· Sierra tau-bench Leaderboard data docs | 78.9% |
| 2 | Grok Voice Think Fast 2.0submission-grok-voice-think-fast-2-0_xai_2026-08-06, usersim-v1.0, effort-high· Sierra tau-bench Leaderboard data docs | 59.6% |
| 3 | GPT Realtime 2submission-gpt-realtime-2_openai_2026-06-29, usersim-v1.0, effort-xhigh· Sierra tau-bench Leaderboard data docs | 47.4% |
| 4 | Qwen3.5 Omni Plus Realtimesubmission-qwen3-5-omni-plus-realtime_qwen_2026-08-06, usersim-v1.0· Sierra tau-bench Leaderboard data docs | 46.5% |
| 5 | Gemini 3.1 Flash Live Previewsubmission-gemini-3-1-flash-live-preview-thinking-high_google_2026-04-02, usersim-v1.0, effort-high, thinking, high· Sierra tau-bench Leaderboard data docs | 45.6% |
| 6 | GPT Realtime 1.5submission-gpt-realtime-1.5_sierra_2026-03-03, usersim-v1.0· Sierra tau-bench Leaderboard data docs | 44.7% |
| 7 | GPT Realtime 2submission-gpt-realtime-2-minimal_openai_2026-07-01, usersim-v1.0, effort-minimal· Sierra tau-bench Leaderboard data docs | 39.5% |
| 8 | gemini-live-2.5-flash-native-audiosubmission-gemini-live-2.5-flash_sierra_2026-03-03, usersim-v1.0· Sierra tau-bench Leaderboard data docs | 29.8% |
| 9 | Gemini 3.1 Flash Live Previewsubmission-gemini-3-1-flash-live-preview-thinking-minimal_google_2026-04-13, usersim-v1.0, effort-minimal, thinking, minimal· Sierra tau-bench Leaderboard data docs | 26.3% |
Data credits: Sierra Research tau2-bench leaderboard submissions (github.com/sierra-research/tau2-bench), MIT (licence)