Skip to content

Benchmark

AHELM: VocalSound - PEM

Scores are percentages.

Models measured
12
Scores recorded
84
Sources
1
IQ weight
0.00

Score distribution

65.5%
Mean
77%
Median
27.4%
Std dev
22.9%
10th percentile
93.8%
90th percentile
0%
Min
0%98.1%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.00 for chat models, measured over 9 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT-4o Audio Previewsound: laughter· AHELM (HELM Audio) data98.1%
2gemini-1.5-pro-001sound: laughter· AHELM (HELM Audio) data96.3%
3Gemini 2.0 Flash Litesound: throatclearing· AHELM (HELM Audio) data95.7%
4Qwen2.5-Omni 7Bsound: laughter· AHELM (HELM Audio) data95.2%
5GPT-4o mini Audio Previewsound: laughter· AHELM (HELM Audio) data94.7%
6Qwen2.5-Omni 7Bsound: sniff· AHELM (HELM Audio) data94.5%
7Gemini 2.0 Flashsound: cough· AHELM (HELM Audio) data94.4%
8Gemini 2.0 Flash Experimentalsound: cough· AHELM (HELM Audio) data94.3%
9gemini-1.5-pro-002sound: laughter· AHELM (HELM Audio) data93.8%
10gemini-1.5-flash-002sound: laughter· AHELM (HELM Audio) data93.6%
11Gemini 2.5 Pro Preview 05-06sound: laughter· AHELM (HELM Audio) data93.1%
12Qwen2.5-Omni 7Bsound: throatclearing· AHELM (HELM Audio) data91.9%
13Gemini 2.0 Flashsound: laughter· AHELM (HELM Audio) data90.6%
14Gemini 2.0 Flash Experimentalsound: laughter· AHELM (HELM Audio) data90.6%
15Qwen2.5-Omni 7B· AHELM (HELM Audio) data90.4%
16Qwen2.5-Omni 7Bsound: cough· AHELM (HELM Audio) data88.2%
17GPT-4o mini Audio Previewsound: throatclearing· AHELM (HELM Audio) data87.8%
18Gemini 2.5 Pro Preview 05-06sound: sigh· AHELM (HELM Audio) data87.4%
19Qwen2.5-Omni 7Bsound: sigh· AHELM (HELM Audio) data86.9%
20GPT-4o Audio Previewsound: sigh· AHELM (HELM Audio) data86.7%
21GPT-4o Audio Previewsound: throatclearing· AHELM (HELM Audio) data86.7%
22Gemini 2.5 Pro Preview 05-06sound: cough· AHELM (HELM Audio) data86.6%
23Gemini 2.5 Pro Preview 05-06· AHELM (HELM Audio) data86%
24GPT-4o mini Audio Previewsound: sigh· AHELM (HELM Audio) data85.8%
25Qwen2.5-Omni 7Bsound: sneeze· AHELM (HELM Audio) data85.4%
26Gemini 2.0 Flash Experimentalsound: sigh· AHELM (HELM Audio) data84.7%
27Gemini 2.0 Flashsound: sigh· AHELM (HELM Audio) data84.6%
28gemini-1.5-flash-001sound: laughter· AHELM (HELM Audio) data84.5%
29Gemini 2.5 Pro Preview 05-06sound: sneeze· AHELM (HELM Audio) data83.8%
30GPT-4o Audio Preview· AHELM (HELM Audio) data83.3%
31Gemini 2.5 Pro Preview 05-06sound: sniff· AHELM (HELM Audio) data82.9%
32Gemini 2.5 Pro Preview 05-06sound: throatclearing· AHELM (HELM Audio) data82.1%
33Gemini 2.5 Flash Preview 05-20 (no think)sound: throatclearing· AHELM (HELM Audio) data81.8%
34gemini-1.5-pro-002sound: cough· AHELM (HELM Audio) data81.5%
35GPT-4o Audio Previewsound: sneeze· AHELM (HELM Audio) data81.4%
36Gemini 2.5 Flash Preview 05-20 (no think)sound: sneeze· AHELM (HELM Audio) data80.8%
37GPT-4o mini Audio Previewsound: cough· AHELM (HELM Audio) data80%
38Gemini 2.0 Flash Litesound: sigh· AHELM (HELM Audio) data79.4%
39GPT-4o mini Audio Preview· AHELM (HELM Audio) data79.4%
40GPT-4o Audio Previewsound: sniff· AHELM (HELM Audio) data77.9%
41Gemini 2.0 Flash Experimentalsound: sniff· AHELM (HELM Audio) data77.6%
42Gemini 2.0 Flashsound: sniff· AHELM (HELM Audio) data77%
43gemini-1.5-pro-001sound: sigh· AHELM (HELM Audio) data76.3%
44GPT-4o mini Audio Previewsound: sneeze· AHELM (HELM Audio) data75.2%
45Gemini 2.0 Flash Litesound: laughter· AHELM (HELM Audio) data73%
46Gemini 2.0 Flash· AHELM (HELM Audio) data71.9%
47Gemini 2.0 Flash Experimental· AHELM (HELM Audio) data71.8%
48GPT-4o Audio Previewsound: cough· AHELM (HELM Audio) data68.9%
49gemini-1.5-pro-002sound: sigh· AHELM (HELM Audio) data66.7%
50Gemini 2.5 Flash Preview 05-20 (no think)sound: laughter· AHELM (HELM Audio) data65.7%

Showing 1–50 of 84 results.