Skip to content

Benchmark

Macedonian LM Evaluation Harness printed average accuracy

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models and systems measured
12
Scores recorded
12
Sources
1
IQ weight
No weight published

Score distribution

0.475
Mean
0.493
Median
0.061
Std dev
0.364
10th percentile
0.54
90th percentile
0.353
Min
0.3530.545
The band covers the middle 80% of scores. The line marks the median.

Results

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
domestic-yak-basetable: 4, task: Average, stage: domestic-yak-base· Macedonian Foundation Model (arXiv:2506.09560v1) data0.52
domestic-yak-instructtable: 4, task: Average, stage: domestic-yak-instruct· Macedonian Foundation Model (arXiv:2506.09560v1) data0.54
domestic-yak†8Btable: 3, task: Avg.· Macedonian Foundation Model (arXiv:2506.09560v1) data0.535
Llama 3.18Btable: 3, task: Avg.· Macedonian Foundation Model (arXiv:2506.09560v1) data0.467
Llama 3.1Llama-3.1-8B-Instructtable: 4, task: Average, stage: Llama 3.1· Macedonian Foundation Model (arXiv:2506.09560v1) data0.47
Llama 3.21Btable: 3, task: Avg.· Macedonian Foundation Model (arXiv:2506.09560v1) data0.353
Llama 3.370Btable: 3, task: Avg.· Macedonian Foundation Model (arXiv:2506.09560v1) data0.545
Mistral7Btable: 3, task: Avg.· Macedonian Foundation Model (arXiv:2506.09560v1) data0.462
Mistral Nemo12Btable: 3, task: Avg.· Macedonian Foundation Model (arXiv:2506.09560v1) data0.493
MKLLM†7Btable: 3, task: Avg.· Macedonian Foundation Model (arXiv:2506.09560v1) data0.51
Phi-3.5-mini3.8Btable: 3, task: Avg.· Macedonian Foundation Model (arXiv:2506.09560v1) data0.364
Qwen2.57Btable: 3, task: Avg.· Macedonian Foundation Model (arXiv:2506.09560v1) data0.439

Data credits: Stefan Krsteski, Matea Tashkovska, Borjan Sazdov, Hristijan Gjoreski and Branislav Gerazov. Towards Open Foundation Language Model and Corpus for Macedonian, arXiv:2506.09560v1. CC BY 4.0 article numeric/protocol extract; restructured, printed values unchanged. No task, corpus, weight or raw survey rights asserted. (licence)