Skip to content

Benchmark

OALL ArabicMMLU

Scores are percentages.

Models measured
51
Scores recorded
51
Sources
1
IQ weight
0.56

Score distribution

47.2%
Mean
48.2%
Median
14.5%
Std dev
30%
10th percentile
67.1%
90th percentile
26.5%
Min
26.5%71.4%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.56 for chat models, measured over 51 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Qwen2.5 72B Instruct· Open Arabic LLM Leaderboard data71.4%
2Gemma 3 27B Pt· Open Arabic LLM Leaderboard data70.8%
3Llama 3.3 70B Instruct· Open Arabic LLM Leaderboard data69.8%
4Qwen2.5 14B· Open Arabic LLM Leaderboard data68.5%
5Qwen2.5 32B Instruct· Open Arabic LLM Leaderboard data68%
6Qwen 2.5 32B Abliterated· Open Arabic LLM Leaderboard data67.1%
7Qwen 2 (72B)· Open Arabic LLM Leaderboard data66.5%
8Gemma 3 27B IT· Open Arabic LLM Leaderboard data65.6%
9Qwen3 8B Base· Open Arabic LLM Leaderboard data65.1%
10Qwen2.5 72B· Open Arabic LLM Leaderboard data64.7%
11Qwen2.5 7B· Open Arabic LLM Leaderboard data61.4%
12Aya Expanse 32B· Open Arabic LLM Leaderboard data60.6%
13Qwen 2.5 14B Instruct· Open Arabic LLM Leaderboard data60%
14Qwen2 7B Instruct· Open Arabic LLM Leaderboard data60%
15Qwen3 4B Base· Open Arabic LLM Leaderboard data59.8%
16Qwen2.5 32B· Open Arabic LLM Leaderboard data59.8%
17C4AI Command R7b Arabic 02 2025· Open Arabic LLM Leaderboard data59.3%
18Aya Expanse 8B· Open Arabic LLM Leaderboard data57.6%
19Llama 3.1 8B Instruct· Open Arabic LLM Leaderboard data53.6%
20Llama 3.1 8B· Open Arabic LLM Leaderboard data52.3%
21Qwen2.5 7B Instruct· Open Arabic LLM Leaderboard data52.3%
22Qwen1.5-14B· Open Arabic LLM Leaderboard data49.7%
23Qwen3 1.7B Base· Open Arabic LLM Leaderboard data49.7%
24Phi 4 Mini· Open Arabic LLM Leaderboard data49.5%
25Qwen2 1.5b Instruct· Open Arabic LLM Leaderboard data49.2%
26Qwen2.5 3B Instruct· Open Arabic LLM Leaderboard data48.2%
27Qwen3 14B Base· Open Arabic LLM Leaderboard data44.6%
28Qwen1.5-7B· Open Arabic LLM Leaderboard data43.9%
29Qwen2.5 1.5B Instruct· Open Arabic LLM Leaderboard data43.5%
30Phi 4· Open Arabic LLM Leaderboard data39.8%
31Qwen1.5-4B· Open Arabic LLM Leaderboard data38.2%
32Gemma 2 27B IT· Open Arabic LLM Leaderboard data37.1%
33qwen2-0.5b-instruct· Open Arabic LLM Leaderboard data36.5%
34Qwen3 8B· Open Arabic LLM Leaderboard data36.4%
35Gemma 2 9B IT· Open Arabic LLM Leaderboard data36.3%
36Qwen3 30B A3B Instruct· Open Arabic LLM Leaderboard data35.9%
37Llama 3.2 3B· Open Arabic LLM Leaderboard data35.7%
38Qwen3 4B Instruct 2507· Open Arabic LLM Leaderboard data31.7%
39Qwen3 32B· Open Arabic LLM Leaderboard data31.5%
40Qwen3 14B· Open Arabic LLM Leaderboard data31.3%
41Qwen3 0.6B· Open Arabic LLM Leaderboard data31.1%
42Qwen2.5 0.5b Instruct· Open Arabic LLM Leaderboard data30.5%
43DeepSeek R1 Distill Qwen 14B· Open Arabic LLM Leaderboard data30.4%
44Qwen3 4B· Open Arabic LLM Leaderboard data30.2%
45Gemma 2 2B IT· Open Arabic LLM Leaderboard data30.1%
46Qwen2.5 1.5B· Open Arabic LLM Leaderboard data30%
47DeepSeek R1 Distill Qwen 32B· Open Arabic LLM Leaderboard data29.8%
48gemma2-27b· Open Arabic LLM Leaderboard data29.8%
49QwQ 32B· Open Arabic LLM Leaderboard data29.8%
50Qwen1.5-1.8B· Open Arabic LLM Leaderboard data26.7%

Showing 1–50 of 51 results.

Data credits: Open Arabic LLM Leaderboard (OALL), Apache-2.0 (licence)