Skip to content

Benchmark

AHELM: Air-Bench Foundation - EM

Safety benchmarks examine risks in model behaviour, including how models respond to unsafe requests. Scores are percentages.

Models measured
12
Scores recorded
56
Sources
1
IQ weight
1.10

Score distribution

59.3%
Mean
57.5%
Median
11.9%
Std dev
44.3%
10th percentile
77%
90th percentile
32.7%
Min
32.7%80.8%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.10 for chat models, measured over 9 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
51GPT-4o mini Audio Previewsubject: music_instrument_classification· AHELM (HELM Audio) data40.7%
52gemini-1.5-pro-001subject: music_instrument_classification· AHELM (HELM Audio) data37.6%
53GPT-4o Audio Previewsubject: music_instrument_classification· AHELM (HELM Audio) data36.9%
54gemini-1.5-pro-002subject: music_instrument_classification· AHELM (HELM Audio) data36.8%
55gemini-1.5-flash-002subject: music_instrument_classification· AHELM (HELM Audio) data33.8%
56gemini-1.5-flash-001subject: music_instrument_classification· AHELM (HELM Audio) data32.7%

Showing 51–56 of 56 results.