Skip to content

Benchmark

Pashto ASR FLEURS ps_af CER (zero-shot)

Scores are percentages. Lower is better.

Models measured
7
Scores recorded
7
Sources
1
IQ weight
1.24

Score distribution

114.2%
Mean
54.2%
Median
100.6%
Std dev
40.8%
10th percentile
142.7%
90th percentile
35.1%
Min
35.1%337.7%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.24 for speech models, measured over 7 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Whisper Large V3table: II, dataset: FLEURS ps_af, metric: CER, protocol: zero-shot· Pashto ASR paper data35.1%
2Whisper Large V3 Turbotable: II, dataset: FLEURS ps_af, metric: CER, protocol: zero-shot· Pashto ASR paper data40.8%
3Whisper Smalltable: II, dataset: FLEURS ps_af, metric: CER, protocol: zero-shot· Pashto ASR paper data49.3%
4Whisper Large v2table: II, dataset: FLEURS ps_af, metric: CER, protocol: zero-shot· Pashto ASR paper data54.2%
5Whisper Mediumtable: II, dataset: FLEURS ps_af, metric: CER, protocol: zero-shot· Pashto ASR paper data139.5%
6Whisper Basetable: II, dataset: FLEURS ps_af, metric: CER, protocol: zero-shot· Pashto ASR paper data142.7%
7Whisper Tinytable: II, dataset: FLEURS ps_af, metric: CER, protocol: zero-shot· Pashto ASR paper data337.7%

Data credits: Hanif Rahman. Benchmarking Multilingual Speech Models on Pashto: Zero-Shot ASR, Script Failure, and Cross-Domain Evaluation, arXiv:2604.04598v1. Numeric/protocol extract from Tables II, III, V and VI; CC BY 4.0 article grant verified. (licence)