Benchmark
MAEB(v1) - AudioReranking
Scores are percentages.
- Models measured
- 11
- Scores recorded
- 11
- Sources
- 1
- IQ weight
- 0.48
Score distribution
- 66.3%
- Mean
- 64.2%
- Median
- 9%
- Std dev
- 56.7%
- 10th percentile
- 83.2%
- 90th percentile
- 55.5%
- Min
This benchmark has a weight of 0.48 for speech models, measured over 5 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.
Leaderboard
| # | Model | Score |
|---|---|---|
| 1 | Jina Embeddings v5 Omni SmallAudioReranking· MTEB Multimodal Boards data docs | 84.4% |
| 2 | Jina Embeddings v5 Omni NanoAudioReranking· MTEB Multimodal Boards data docs | 83.2% |
| 3 | Whisper MediumAudioReranking· MTEB Multimodal Boards data docs | 67.6% |
| 4 | embeddinggemma-2AudioReranking· MTEB Multimodal Boards data docs | 66.5% |
| 5 | Whisper BaseAudioReranking· MTEB Multimodal Boards data docs | 65% |
| 6 | Whisper SmallAudioReranking· MTEB Multimodal Boards data docs | 64.2% |
| 7 | Whisper Large V3AudioReranking· MTEB Multimodal Boards data docs | 63.9% |
| 8 | Whisper TinyAudioReranking· MTEB Multimodal Boards data docs | 63.4% |
| 9 | Qwen2.5-Omni 7BAudioReranking· MTEB Multimodal Boards data docs | 58.9% |
| 10 | Qwen3 Omni 30B A3B ThinkingAudioReranking· MTEB Multimodal Boards data docs | 56.7% |
| 11 | qwen3-omni-30b-a3b-instructAudioReranking· MTEB Multimodal Boards data docs | 55.5% |