Skip to content

Benchmark

MAEB(v1) - AudioMultilabelClassification

Scores are percentages.

Models measured
11
Scores recorded
11
Sources
1
IQ weight
0.57

Score distribution

41.8%
Mean
46.3%
Median
20.3%
Std dev
13%
10th percentile
69.3%
90th percentile
9.8%
Min
9.8%70.8%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.57 for speech models, measured over 5 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Jina Embeddings v5 Omni NanoAudioMultilabelClassification· MTEB Multimodal Boards data docs70.8%
2Jina Embeddings v5 Omni SmallAudioMultilabelClassification· MTEB Multimodal Boards data docs69.3%
3Whisper MediumAudioMultilabelClassification· MTEB Multimodal Boards data docs56.8%
4Whisper SmallAudioMultilabelClassification· MTEB Multimodal Boards data docs51.4%
5Whisper Large V3AudioMultilabelClassification· MTEB Multimodal Boards data docs47.6%
6Whisper BaseAudioMultilabelClassification· MTEB Multimodal Boards data docs46.3%
7Whisper TinyAudioMultilabelClassification· MTEB Multimodal Boards data docs41.4%
8embeddinggemma-2AudioMultilabelClassification· MTEB Multimodal Boards data docs38.8%
9Qwen3 Omni 30B A3B ThinkingAudioMultilabelClassification· MTEB Multimodal Boards data docs15.2%
10qwen3-omni-30b-a3b-instructAudioMultilabelClassification· MTEB Multimodal Boards data docs13%
11Qwen2.5-Omni 7BAudioMultilabelClassification· MTEB Multimodal Boards data docs9.8%