Skip to content

Benchmark

AHELM: VocalSound - PEM

Scores are percentages.

Models measured
12
Scores recorded
84
Sources
1
IQ weight
0.00

Score distribution

65.5%
Mean
77%
Median
27.4%
Std dev
22.9%
10th percentile
93.8%
90th percentile
0%
Min
0%98.1%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.00 for chat models, measured over 9 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
51gemini-1.5-flash-001sound: sigh· AHELM (HELM Audio) data64.6%
52gemini-1.5-flash-002sound: sigh· AHELM (HELM Audio) data62.7%
53Gemini 2.5 Flash Preview 05-20 (no think)· AHELM (HELM Audio) data62.6%
54gemini-1.5-flash-001sound: throatclearing· AHELM (HELM Audio) data62.1%
55Gemini 2.0 Flashsound: sneeze· AHELM (HELM Audio) data61.3%
56Gemini 2.5 Flash Preview 05-20 (no think)sound: cough· AHELM (HELM Audio) data61.1%
57Gemini 2.0 Flash Experimentalsound: sneeze· AHELM (HELM Audio) data60.7%
58Gemini 2.5 Flash Preview 05-20 (no think)sound: sigh· AHELM (HELM Audio) data58.4%
59gemini-1.5-pro-001sound: sneeze· AHELM (HELM Audio) data54%
60gemini-1.5-pro-002sound: sneeze· AHELM (HELM Audio) data53.1%
61GPT-4o mini Audio Previewsound: sniff· AHELM (HELM Audio) data52.9%
62gemini-1.5-pro-002· AHELM (HELM Audio) data52.8%
63Gemini 2.0 Flash Lite· AHELM (HELM Audio) data50.6%
64gemini-1.5-pro-001· AHELM (HELM Audio) data49.2%
65gemini-1.5-flash-001· AHELM (HELM Audio) data46.3%
66gemini-1.5-flash-001sound: cough· AHELM (HELM Audio) data43.4%
67gemini-1.5-pro-001sound: cough· AHELM (HELM Audio) data43%
68gemini-1.5-flash-002sound: throatclearing· AHELM (HELM Audio) data42.6%
69gemini-1.5-flash-002· AHELM (HELM Audio) data41.8%
70gemini-1.5-flash-002sound: cough· AHELM (HELM Audio) data39.7%
71Gemini 2.0 Flash Litesound: sneeze· AHELM (HELM Audio) data39.6%
72Gemini 2.5 Flash Preview 05-20 (no think)sound: sniff· AHELM (HELM Audio) data27.7%
73gemini-1.5-pro-001sound: throatclearing· AHELM (HELM Audio) data25%
74Gemini 2.0 Flashsound: throatclearing· AHELM (HELM Audio) data23.5%
75gemini-1.5-flash-001sound: sneeze· AHELM (HELM Audio) data23.5%
76Gemini 2.0 Flash Experimentalsound: throatclearing· AHELM (HELM Audio) data22.9%
77gemini-1.5-pro-002sound: throatclearing· AHELM (HELM Audio) data21.5%
78Gemini 2.0 Flash Litesound: cough· AHELM (HELM Audio) data14.6%
79gemini-1.5-flash-002sound: sneeze· AHELM (HELM Audio) data12.1%
80Gemini 2.0 Flash Litesound: sniff· AHELM (HELM Audio) data1.4%
81gemini-1.5-pro-001sound: sniff· AHELM (HELM Audio) data0.8%
82gemini-1.5-pro-002sound: sniff· AHELM (HELM Audio) data0.3%
83gemini-1.5-flash-002sound: sniff· AHELM (HELM Audio) data0.1%
84gemini-1.5-flash-001sound: sniff· AHELM (HELM Audio) data0%

Showing 51–84 of 84 results.