Skip to content

Benchmark

AHELM: Robust Speech Bench - WER

Scores are percentages. Lower is better.

Models measured
12
Scores recorded
132
Sources
1
IQ weight
1.49

Score distribution

0.287%
Mean
0.103%
Median
0.453%
Std dev
0.044%
10th percentile
0.724%
90th percentile
0.0387%
Min
0.0387%3.5%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.49 for chat models, measured over 9 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
51Gemini 2.5 Flash Preview 05-20 (no think)subject: librispeech_env_noise, level: 1· AHELM (HELM Audio) data0.0766%
52Gemini 2.5 Flash Preview 05-20 (no think)subject: librispeech_env_noise, level: 2· AHELM (HELM Audio) data0.0766%
53Gemini 2.5 Flash Preview 05-20 (no think)subject: librispeech_env_noise, level: 3· AHELM (HELM Audio) data0.0766%
54Gemini 2.5 Flash Preview 05-20 (no think)subject: librispeech_gnoise, level: 1· AHELM (HELM Audio) data0.0766%
55Gemini 2.5 Flash Preview 05-20 (no think)subject: librispeech_gnoise, level: 2· AHELM (HELM Audio) data0.0766%
56Gemini 2.5 Flash Preview 05-20 (no think)subject: librispeech_gnoise, level: 3· AHELM (HELM Audio) data0.0766%
57gemini-1.5-pro-002subject: accented_cv· AHELM (HELM Audio) data0.0848%
58Gemini 2.0 Flash Experimentalsubject: accented_cv· AHELM (HELM Audio) data0.0921%
59Gemini 2.0 Flashsubject: accented_cv· AHELM (HELM Audio) data0.0928%
60gemini-1.5-flash-002subject: accented_cv· AHELM (HELM Audio) data0.101%
61Qwen2.5-Omni 7B· AHELM (HELM Audio) data0.103%
62Qwen2.5-Omni 7Bsubject: librispeech_env_noise, level: 1· AHELM (HELM Audio) data0.103%
63Qwen2.5-Omni 7Bsubject: librispeech_env_noise, level: 2· AHELM (HELM Audio) data0.103%
64Qwen2.5-Omni 7Bsubject: librispeech_env_noise, level: 3· AHELM (HELM Audio) data0.103%
65Qwen2.5-Omni 7Bsubject: librispeech_gnoise, level: 1· AHELM (HELM Audio) data0.103%
66Qwen2.5-Omni 7Bsubject: librispeech_gnoise, level: 2· AHELM (HELM Audio) data0.103%
67Qwen2.5-Omni 7Bsubject: librispeech_gnoise, level: 3· AHELM (HELM Audio) data0.103%
68gemini-1.5-pro-001subject: accented_cv· AHELM (HELM Audio) data0.104%
69GPT-4o Audio Previewsubject: librispeech_gnoise, level: 1· AHELM (HELM Audio) data0.113%
70GPT-4o Audio Previewsubject: librispeech_gnoise, level: 2· AHELM (HELM Audio) data0.113%
71GPT-4o Audio Previewsubject: librispeech_gnoise, level: 3· AHELM (HELM Audio) data0.113%
72GPT-4o Audio Previewsubject: librispeech_env_noise, level: 1· AHELM (HELM Audio) data0.113%
73GPT-4o Audio Previewsubject: librispeech_env_noise, level: 2· AHELM (HELM Audio) data0.113%
74GPT-4o Audio Previewsubject: librispeech_env_noise, level: 3· AHELM (HELM Audio) data0.113%
75GPT-4o Audio Previewsubject: accented_cv· AHELM (HELM Audio) data0.131%
76GPT-4o mini Audio Previewsubject: accented_cv_es· AHELM (HELM Audio) data0.137%
77Gemini 2.0 Flash Experimentalsubject: ami_near· AHELM (HELM Audio) data0.16%
78GPT-4o mini Audio Previewsubject: librispeech_gnoise, level: 1· AHELM (HELM Audio) data0.163%
79GPT-4o mini Audio Previewsubject: librispeech_gnoise, level: 2· AHELM (HELM Audio) data0.163%
80GPT-4o mini Audio Previewsubject: librispeech_gnoise, level: 3· AHELM (HELM Audio) data0.163%
81Gemini 2.0 Flashsubject: ami_near· AHELM (HELM Audio) data0.163%
82GPT-4o mini Audio Previewsubject: librispeech_env_noise, level: 1· AHELM (HELM Audio) data0.166%
83GPT-4o mini Audio Previewsubject: librispeech_env_noise, level: 2· AHELM (HELM Audio) data0.166%
84GPT-4o mini Audio Previewsubject: librispeech_env_noise, level: 3· AHELM (HELM Audio) data0.166%
85Gemini 2.0 Flash Experimental· AHELM (HELM Audio) data0.171%
86Gemini 2.0 Flash· AHELM (HELM Audio) data0.178%
87GPT-4o mini Audio Previewsubject: accented_cv· AHELM (HELM Audio) data0.18%
88gemini-1.5-flash-001subject: ami_near· AHELM (HELM Audio) data0.199%
89gemini-1.5-pro-002· AHELM (HELM Audio) data0.207%
90gemini-1.5-pro-002subject: ami_near· AHELM (HELM Audio) data0.207%
91gemini-1.5-pro-001· AHELM (HELM Audio) data0.213%
92gemini-1.5-flash-002· AHELM (HELM Audio) data0.214%
93gemini-1.5-flash-002subject: ami_near· AHELM (HELM Audio) data0.235%
94GPT-4o Audio Previewsubject: accented_cv_es· AHELM (HELM Audio) data0.245%
95gemini-1.5-pro-001subject: ami_near· AHELM (HELM Audio) data0.257%
96gemini-1.5-flash-001subject: librispeech_env_noise, level: 2· AHELM (HELM Audio) data0.359%
97gemini-1.5-flash-001subject: librispeech_env_noise, level: 3· AHELM (HELM Audio) data0.36%
98gemini-1.5-flash-001subject: librispeech_env_noise, level: 1· AHELM (HELM Audio) data0.36%
99gemini-1.5-flash-001subject: librispeech_gnoise, level: 1· AHELM (HELM Audio) data0.361%
100gemini-1.5-flash-001subject: librispeech_gnoise, level: 2· AHELM (HELM Audio) data0.361%

Showing 51–100 of 132 results.