Skip to content

Benchmark

Pashto ASR FLEURS ps_af script Ar/Da/Ur (zero-shot)

Scores are percentages.

Models measured
7
Scores recorded
7
Sources
1
IQ weight
No weight published

Score distribution

63.6%
Mean
71.5%
Median
35%
Std dev
29.5%
10th percentile
98.8%
90th percentile
6.4%
Min
6.4%99.2%
The band covers the middle 80% of scores. The line marks the median.

Results

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

ModelScore
Whisper Basetable: III, dataset: FLEURS ps_af, metric: script Ar/Da/Ur, protocol: zero-shot· Pashto ASR paper data42.6%
Whisper Large v2table: III, dataset: FLEURS ps_af, metric: script Ar/Da/Ur, protocol: zero-shot· Pashto ASR paper data71.5%
Whisper Large V3table: III, dataset: FLEURS ps_af, metric: script Ar/Da/Ur, protocol: zero-shot· Pashto ASR paper data99.2%
Whisper Large V3 Turbotable: III, dataset: FLEURS ps_af, metric: script Ar/Da/Ur, protocol: zero-shot· Pashto ASR paper data98.8%
Whisper Mediumtable: III, dataset: FLEURS ps_af, metric: script Ar/Da/Ur, protocol: zero-shot· Pashto ASR paper data29.5%
Whisper Smalltable: III, dataset: FLEURS ps_af, metric: script Ar/Da/Ur, protocol: zero-shot· Pashto ASR paper data97.5%
Whisper Tinytable: III, dataset: FLEURS ps_af, metric: script Ar/Da/Ur, protocol: zero-shot· Pashto ASR paper data6.4%

Data credits: Hanif Rahman. Benchmarking Multilingual Speech Models on Pashto: Zero-Shot ASR, Script Failure, and Cross-Domain Evaluation, arXiv:2604.04598v1. Numeric/protocol extract from Tables II, III, V and VI; CC BY 4.0 article grant verified. (licence)