Skip to content

Benchmark

AfriHG XL-Sum summarization Tigrinya (tir) ROUGE-2

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models and systems measured
1
Scores recorded
1
Sources
1
IQ weight
No weight published

Score distribution

8
Mean
8
Median
0
Std dev
8
10th percentile
8
90th percentile
8
Min

Results

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
mT5-baseXL-Sum summarization (Hasan et al., 2021)table: 1, system: mT5-base, task: summary, language: tir, metric: ROUGE-2· AfriHG (arXiv:2412.20223v1) data8

Data credits: Toyib Ogunremi, Serah Akojenu, Anthony Soronnadi, Olubayo Adekanmbi and David Ifeoluwa Adelani. AfriHG: News headline generation for African Languages, arXiv:2412.20223v1. CC BY 4.0 article numeric/protocol extract, restructured with values unchanged. Relayed summarization scores credited to Hasan et al. (2021); no task or model rights asserted. (licence)