Skip to content

Benchmark

MAEB(v1) - AudioPairClassification

Scores are percentages.

Models measured
11
Scores recorded
11
Sources
1
IQ weight
0.83

Score distribution

56.2%
Mean
52.9%
Median
6.1%
Std dev
51.5%
10th percentile
65.6%
90th percentile
50.8%
Min
50.8%67.5%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.83 for speech models, measured over 5 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1embeddinggemma-2AudioPairClassification· MTEB Multimodal Boards data docs67.5%
2Jina Embeddings v5 Omni NanoAudioPairClassification· MTEB Multimodal Boards data docs65.6%
3Jina Embeddings v5 Omni SmallAudioPairClassification· MTEB Multimodal Boards data docs65.4%
4Whisper MediumAudioPairClassification· MTEB Multimodal Boards data docs53.9%
5qwen3-omni-30b-a3b-instructAudioPairClassification· MTEB Multimodal Boards data docs53.9%
6Qwen2.5-Omni 7BAudioPairClassification· MTEB Multimodal Boards data docs52.9%
7Whisper SmallAudioPairClassification· MTEB Multimodal Boards data docs52.6%
8Whisper Large V3AudioPairClassification· MTEB Multimodal Boards data docs52.5%
9Whisper BaseAudioPairClassification· MTEB Multimodal Boards data docs52.1%
10Whisper TinyAudioPairClassification· MTEB Multimodal Boards data docs51.5%
11Qwen3 Omni 30B A3B ThinkingAudioPairClassification· MTEB Multimodal Boards data docs50.8%