Benchmark
MAEB(v1, audio-only) - AudioClassification
Scores are percentages.
- Models measured
- 8
- Scores recorded
- 8
- Sources
- 1
- IQ weight
- 0.57
Score distribution
- 50.8%
- Mean
- 51.3%
- Median
- 2.3%
- Std dev
- 48.3%
- 10th percentile
- 52.3%
- 90th percentile
- 47.3%
- Min
This benchmark has a weight of 0.57 for speech models, measured over 5 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.
Leaderboard
| # | Model | Score |
|---|---|---|
| 1 | Whisper MediumAudioClassification· MTEB Multimodal Boards data docs | 54.9% |
| 2 | embeddinggemma-2AudioClassification· MTEB Multimodal Boards data docs | 52.3% |
| 3 | Jina Embeddings v5 Omni SmallAudioClassification· MTEB Multimodal Boards data docs | 52% |
| 4 | Jina Embeddings v5 Omni NanoAudioClassification· MTEB Multimodal Boards data docs | 51.3% |
| 5 | Whisper SmallAudioClassification· MTEB Multimodal Boards data docs | 50.4% |
| 6 | Whisper BaseAudioClassification· MTEB Multimodal Boards data docs | 49.7% |
| 7 | Whisper Large V3AudioClassification· MTEB Multimodal Boards data docs | 48.3% |
| 8 | Whisper TinyAudioClassification· MTEB Multimodal Boards data docs | 47.3% |