Skip to content

Benchmark

AHELM: Robust Speech Bench - WER

Scores are percentages. Lower is better.

Models measured
12
Scores recorded
132
Sources
1
IQ weight
1.49

Score distribution

0.287%
Mean
0.103%
Median
0.453%
Std dev
0.044%
10th percentile
0.724%
90th percentile
0.0387%
Min
0.0387%3.5%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.49 for chat models, measured over 9 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemini 2.5 Pro Preview 05-06subject: librispeech_env_noise, level: 1· AHELM (HELM Audio) data0.0387%
2Gemini 2.5 Pro Preview 05-06subject: librispeech_env_noise, level: 2· AHELM (HELM Audio) data0.0387%
3Gemini 2.5 Pro Preview 05-06subject: librispeech_env_noise, level: 3· AHELM (HELM Audio) data0.0387%
4Gemini 2.5 Pro Preview 05-06· AHELM (HELM Audio) data0.0388%
5Gemini 2.5 Pro Preview 05-06subject: librispeech_gnoise, level: 1· AHELM (HELM Audio) data0.0388%
6Gemini 2.5 Pro Preview 05-06subject: librispeech_gnoise, level: 2· AHELM (HELM Audio) data0.0388%
7Gemini 2.5 Pro Preview 05-06subject: librispeech_gnoise, level: 3· AHELM (HELM Audio) data0.0388%
8Gemini 2.0 Flashsubject: librispeech_env_noise, level: 2· AHELM (HELM Audio) data0.0435%
9Gemini 2.0 Flashsubject: librispeech_env_noise, level: 3· AHELM (HELM Audio) data0.0435%
10Gemini 2.0 Flashsubject: librispeech_env_noise, level: 1· AHELM (HELM Audio) data0.0436%
11Gemini 2.0 Flashsubject: librispeech_gnoise, level: 1· AHELM (HELM Audio) data0.0437%
12Gemini 2.0 Flashsubject: librispeech_gnoise, level: 2· AHELM (HELM Audio) data0.0438%
13Gemini 2.0 Flashsubject: librispeech_gnoise, level: 3· AHELM (HELM Audio) data0.0439%
14Gemini 2.0 Flash Experimentalsubject: librispeech_env_noise, level: 1· AHELM (HELM Audio) data0.0443%
15Gemini 2.0 Flash Experimentalsubject: librispeech_env_noise, level: 3· AHELM (HELM Audio) data0.0443%
16Gemini 2.0 Flash Experimentalsubject: librispeech_gnoise, level: 1· AHELM (HELM Audio) data0.0444%
17Gemini 2.0 Flash Experimentalsubject: librispeech_gnoise, level: 2· AHELM (HELM Audio) data0.0444%
18Gemini 2.0 Flash Experimentalsubject: librispeech_gnoise, level: 3· AHELM (HELM Audio) data0.0444%
19Gemini 2.0 Flash Experimentalsubject: librispeech_env_noise, level: 2· AHELM (HELM Audio) data0.0446%
20Gemini 2.0 Flash Litesubject: librispeech_env_noise, level: 2· AHELM (HELM Audio) data0.0491%
21Gemini 2.0 Flash Litesubject: librispeech_gnoise, level: 1· AHELM (HELM Audio) data0.0492%
22Gemini 2.0 Flash Litesubject: librispeech_gnoise, level: 2· AHELM (HELM Audio) data0.0492%
23Gemini 2.0 Flash Litesubject: librispeech_gnoise, level: 3· AHELM (HELM Audio) data0.0492%
24Gemini 2.0 Flash Lite· AHELM (HELM Audio) data0.0492%
25Gemini 2.0 Flash Litesubject: librispeech_env_noise, level: 1· AHELM (HELM Audio) data0.0492%
26Gemini 2.0 Flash Litesubject: librispeech_env_noise, level: 3· AHELM (HELM Audio) data0.0492%
27gemini-1.5-pro-002subject: librispeech_gnoise, level: 3· AHELM (HELM Audio) data0.0528%
28gemini-1.5-pro-002subject: librispeech_gnoise, level: 2· AHELM (HELM Audio) data0.0529%
29gemini-1.5-pro-002subject: librispeech_gnoise, level: 1· AHELM (HELM Audio) data0.0536%
30gemini-1.5-pro-002subject: librispeech_env_noise, level: 2· AHELM (HELM Audio) data0.0549%
31gemini-1.5-pro-002subject: librispeech_env_noise, level: 3· AHELM (HELM Audio) data0.0549%
32gemini-1.5-pro-002subject: librispeech_env_noise, level: 1· AHELM (HELM Audio) data0.0551%
33gemini-1.5-flash-002subject: librispeech_env_noise, level: 1· AHELM (HELM Audio) data0.0617%
34gemini-1.5-flash-002subject: librispeech_env_noise, level: 2· AHELM (HELM Audio) data0.0619%
35Gemini 2.0 Flashsubject: accented_cv_es· AHELM (HELM Audio) data0.0627%
36gemini-1.5-flash-002subject: librispeech_env_noise, level: 3· AHELM (HELM Audio) data0.0628%
37gemini-1.5-flash-002subject: librispeech_gnoise, level: 1· AHELM (HELM Audio) data0.0631%
38gemini-1.5-flash-002subject: librispeech_gnoise, level: 2· AHELM (HELM Audio) data0.0631%
39gemini-1.5-flash-002subject: librispeech_gnoise, level: 3· AHELM (HELM Audio) data0.0631%
40gemini-1.5-pro-002subject: accented_cv_es· AHELM (HELM Audio) data0.0634%
41Gemini 2.0 Flash Experimentalsubject: accented_cv_es· AHELM (HELM Audio) data0.0636%
42gemini-1.5-pro-001subject: librispeech_gnoise, level: 1· AHELM (HELM Audio) data0.0693%
43gemini-1.5-pro-001subject: librispeech_env_noise, level: 3· AHELM (HELM Audio) data0.0697%
44gemini-1.5-pro-001subject: librispeech_gnoise, level: 2· AHELM (HELM Audio) data0.07%
45gemini-1.5-pro-001subject: librispeech_env_noise, level: 2· AHELM (HELM Audio) data0.07%
46gemini-1.5-flash-002subject: accented_cv_es· AHELM (HELM Audio) data0.0702%
47gemini-1.5-pro-001subject: librispeech_env_noise, level: 1· AHELM (HELM Audio) data0.0705%
48gemini-1.5-pro-001subject: librispeech_gnoise, level: 3· AHELM (HELM Audio) data0.0707%
49gemini-1.5-pro-001subject: accented_cv_es· AHELM (HELM Audio) data0.0758%
50Gemini 2.5 Flash Preview 05-20 (no think)· AHELM (HELM Audio) data0.0766%

Showing 1–50 of 132 results.