Skip to content

Benchmark

AHELM: PARADE - EM

Scores are percentages.

Models measured
12
Scores recorded
60
Sources
1
IQ weight
0.38

Score distribution

46.5%
Mean
46%
Median
26.2%
Std dev
9.8%
10th percentile
83%
90th percentile
2.7%
Min
2.7%97%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.38 for chat models, measured over 9 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT-4o mini Audio Previewsubset: status, voice: male· AHELM (HELM Audio) data97%
2GPT-4o mini Audio Previewsubset: status, voice: female· AHELM (HELM Audio) data95%
3GPT-4o Audio Previewsubset: status, voice: female· AHELM (HELM Audio) data91%
4GPT-4o Audio Previewsubset: status, voice: male· AHELM (HELM Audio) data90%
5GPT-4o mini Audio Preview· AHELM (HELM Audio) data85.7%
6GPT-4o Audio Preview· AHELM (HELM Audio) data84.7%
7Qwen2.5-Omni 7Bsubset: status, voice: female· AHELM (HELM Audio) data83%
8Qwen2.5-Omni 7Bsubset: status, voice: male· AHELM (HELM Audio) data81%
9GPT-4o Audio Previewsubset: occupation, voice: female· AHELM (HELM Audio) data79.1%
10GPT-4o Audio Previewsubset: occupation, voice: male· AHELM (HELM Audio) data78.6%
11GPT-4o mini Audio Previewsubset: occupation, voice: male· AHELM (HELM Audio) data76.2%
12GPT-4o mini Audio Previewsubset: occupation, voice: female· AHELM (HELM Audio) data74.5%
13Gemini 2.0 Flash Experimentalsubset: status, voice: female· AHELM (HELM Audio) data69%
14Gemini 2.0 Flashsubset: status, voice: female· AHELM (HELM Audio) data68%
15Gemini 2.5 Flash Preview 05-20 (no think)subset: status, voice: male· AHELM (HELM Audio) data68%
16Gemini 2.5 Flash Preview 05-20 (no think)subset: status, voice: female· AHELM (HELM Audio) data66%
17Qwen2.5-Omni 7B· AHELM (HELM Audio) data63.4%
18Gemini 2.0 Flash Litesubset: status, voice: female· AHELM (HELM Audio) data63%
19Gemini 2.0 Flashsubset: status, voice: male· AHELM (HELM Audio) data61%
20Gemini 2.0 Flash Experimentalsubset: status, voice: male· AHELM (HELM Audio) data61%
21Gemini 2.0 Flash Litesubset: status, voice: male· AHELM (HELM Audio) data60%
22gemini-1.5-flash-001subset: status, voice: male· AHELM (HELM Audio) data56%
23gemini-1.5-flash-001subset: status, voice: female· AHELM (HELM Audio) data55%
24Gemini 2.5 Pro Preview 05-06subset: status, voice: female· AHELM (HELM Audio) data54%
25gemini-1.5-flash-002subset: status, voice: female· AHELM (HELM Audio) data53%
26gemini-1.5-flash-002subset: status, voice: male· AHELM (HELM Audio) data52%
27Gemini 2.5 Flash Preview 05-20 (no think)· AHELM (HELM Audio) data51.4%
28Gemini 2.0 Flash Experimental· AHELM (HELM Audio) data46.5%
29Gemini 2.0 Flash· AHELM (HELM Audio) data46.3%
30Gemini 2.5 Pro Preview 05-06subset: status, voice: male· AHELM (HELM Audio) data46%
31Qwen2.5-Omni 7Bsubset: occupation, voice: male· AHELM (HELM Audio) data45%
32Qwen2.5-Omni 7Bsubset: occupation, voice: female· AHELM (HELM Audio) data44.4%
33Gemini 2.0 Flash Lite· AHELM (HELM Audio) data43.6%
34gemini-1.5-pro-002subset: status, voice: male· AHELM (HELM Audio) data42%
35Gemini 2.5 Flash Preview 05-20 (no think)subset: occupation, voice: male· AHELM (HELM Audio) data37.1%
36gemini-1.5-pro-001subset: status, voice: female· AHELM (HELM Audio) data35%
37gemini-1.5-pro-002subset: status, voice: female· AHELM (HELM Audio) data35%
38Gemini 2.5 Flash Preview 05-20 (no think)subset: occupation, voice: female· AHELM (HELM Audio) data34.4%
39gemini-1.5-pro-001subset: status, voice: male· AHELM (HELM Audio) data33%
40Gemini 2.5 Pro Preview 05-06· AHELM (HELM Audio) data32.4%
41gemini-1.5-flash-002· AHELM (HELM Audio) data31.2%
42gemini-1.5-flash-001· AHELM (HELM Audio) data29.2%
43Gemini 2.0 Flashsubset: occupation, voice: female· AHELM (HELM Audio) data28.7%
44Gemini 2.0 Flash Experimentalsubset: occupation, voice: female· AHELM (HELM Audio) data28.7%
45Gemini 2.0 Flash Litesubset: occupation, voice: female· AHELM (HELM Audio) data28.5%
46Gemini 2.0 Flashsubset: occupation, voice: male· AHELM (HELM Audio) data27.6%
47Gemini 2.0 Flash Experimentalsubset: occupation, voice: male· AHELM (HELM Audio) data27.4%
48Gemini 2.0 Flash Litesubset: occupation, voice: male· AHELM (HELM Audio) data23%
49gemini-1.5-pro-001· AHELM (HELM Audio) data21.7%
50gemini-1.5-pro-002· AHELM (HELM Audio) data21.5%

Showing 1–50 of 60 results.