Skip to content

Benchmark

AHELM: Robust Speech Bench - WER

Scores are percentages. Lower is better.

Models measured
12
Scores recorded
132
Sources
1
IQ weight
1.49

Score distribution

0.287%
Mean
0.103%
Median
0.453%
Std dev
0.044%
10th percentile
0.724%
90th percentile
0.0387%
Min
0.0387%3.5%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.49 for chat models, measured over 9 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
101gemini-1.5-flash-001subject: librispeech_gnoise, level: 3· AHELM (HELM Audio) data0.361%
102Gemini 2.0 Flash Experimentalsubject: ami_far· AHELM (HELM Audio) data0.373%
103Gemini 2.0 Flashsubject: ami_far· AHELM (HELM Audio) data0.399%
104Gemini 2.0 Flash Experimentalsubject: chime_near· AHELM (HELM Audio) data0.468%
105GPT-4o mini Audio Preview· AHELM (HELM Audio) data0.471%
106gemini-1.5-pro-002subject: ami_far· AHELM (HELM Audio) data0.491%
107gemini-1.5-flash-002subject: ami_far· AHELM (HELM Audio) data0.491%
108gemini-1.5-flash-001· AHELM (HELM Audio) data0.498%
109gemini-1.5-pro-001subject: ami_far· AHELM (HELM Audio) data0.499%
110gemini-1.5-pro-001subject: chime_near· AHELM (HELM Audio) data0.515%
111gemini-1.5-flash-001subject: ami_far· AHELM (HELM Audio) data0.516%
112Gemini 2.0 Flashsubject: chime_near· AHELM (HELM Audio) data0.518%
113gemini-1.5-flash-002subject: chime_near· AHELM (HELM Audio) data0.554%
114gemini-1.5-pro-002subject: chime_near· AHELM (HELM Audio) data0.579%
115Gemini 2.0 Flash Experimentalsubject: chime_far· AHELM (HELM Audio) data0.632%
116Gemini 2.0 Flashsubject: chime_far· AHELM (HELM Audio) data0.633%
117gemini-1.5-flash-001subject: accented_cv· AHELM (HELM Audio) data0.675%
118gemini-1.5-pro-001subject: chime_far· AHELM (HELM Audio) data0.68%
119gemini-1.5-flash-001subject: accented_cv_es· AHELM (HELM Audio) data0.724%
120gemini-1.5-pro-002subject: chime_far· AHELM (HELM Audio) data0.735%
121gemini-1.5-flash-002subject: chime_far· AHELM (HELM Audio) data0.742%
122GPT-4o mini Audio Previewsubject: ami_near· AHELM (HELM Audio) data0.743%
123GPT-4o mini Audio Previewsubject: ami_far· AHELM (HELM Audio) data0.809%
124GPT-4o Audio Preview· AHELM (HELM Audio) data0.822%
125gemini-1.5-flash-001subject: chime_far· AHELM (HELM Audio) data0.848%
126gemini-1.5-flash-001subject: chime_near· AHELM (HELM Audio) data0.858%
127GPT-4o mini Audio Previewsubject: chime_near· AHELM (HELM Audio) data1.2%
128GPT-4o Audio Previewsubject: ami_near· AHELM (HELM Audio) data1.4%
129GPT-4o Audio Previewsubject: ami_far· AHELM (HELM Audio) data1.5%
130GPT-4o mini Audio Previewsubject: chime_far· AHELM (HELM Audio) data1.6%
131GPT-4o Audio Previewsubject: chime_near· AHELM (HELM Audio) data2.4%
132GPT-4o Audio Previewsubject: chime_far· AHELM (HELM Audio) data3.5%

Showing 101–132 of 132 results.