Skip to content

Benchmark

PashtoBench factual QA English-minus-Pashto delta (token F-measure)

Multilingual benchmarks compare model performance across languages.

Models measured
4
Scores recorded
4
Sources
1
IQ weight
No weight published

Score distribution

0
Mean
0.013
Median
0.039
Std dev
-0.062
10th percentile
0.046
90th percentile
-0.062
Min
-0.0620.046
The band covers the middle 80% of scores. The line marks the median.

Results

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

ModelScore
Claude Sonnet 5column: F1 delta· PashtoBench v1.0 data docs0.002
GPT 4ocolumn: F1 delta· PashtoBench v1.0 data docs0.013
Llama 3.3 70B Instructcolumn: F1 delta· PashtoBench v1.0 data docs-0.062
Qwen2.5 72B Instructcolumn: F1 delta· PashtoBench v1.0 data docs0.046

Data credits: PashtoBench v1.0, Jawad Noori. Public results/scores numbers; code MIT and data/ CC BY 4.0 grants do not name results/, so no results licence is asserted.