Skip to content

Benchmark

OALL ArbMMLU-HT

Scores are percentages.

Models measured
51
Scores recorded
51
Sources
1
IQ weight
0.65

Score distribution

42.5%
Mean
39.1%
Median
15.8%
Std dev
23.6%
10th percentile
66.4%
90th percentile
23.3%
Min
23.3%72.2%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.65 for chat models, measured over 51 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Qwen2.5 72B Instruct· Open Arabic LLM Leaderboard data72.2%
2Qwen 2 (72B)· Open Arabic LLM Leaderboard data68.1%
3Llama 3.3 70B Instruct· Open Arabic LLM Leaderboard data67.7%
4Gemma 3 27B IT· Open Arabic LLM Leaderboard data66.9%
5Gemma 3 27B Pt· Open Arabic LLM Leaderboard data66.6%
6Qwen2.5 72B· Open Arabic LLM Leaderboard data66.4%
7Qwen2.5 32B Instruct· Open Arabic LLM Leaderboard data65.7%
8Qwen 2.5 32B Abliterated· Open Arabic LLM Leaderboard data65.6%
9Qwen2.5 14B· Open Arabic LLM Leaderboard data61.7%
10Qwen3 8B Base· Open Arabic LLM Leaderboard data61.5%
11Gemma 2 27B IT· Open Arabic LLM Leaderboard data61.2%
12Aya Expanse 32B· Open Arabic LLM Leaderboard data58.9%
13Qwen2.5 32B· Open Arabic LLM Leaderboard data56.2%
14Qwen 2.5 14B Instruct· Open Arabic LLM Leaderboard data55.7%
15Qwen3 4B Base· Open Arabic LLM Leaderboard data53.7%
16Qwen2.5 7B· Open Arabic LLM Leaderboard data51.7%
17Qwen2 7B Instruct· Open Arabic LLM Leaderboard data51.3%
18C4AI Command R7b Arabic 02 2025· Open Arabic LLM Leaderboard data50.1%
19Aya Expanse 8B· Open Arabic LLM Leaderboard data49.2%
20Llama 3.1 8B Instruct· Open Arabic LLM Leaderboard data47.9%
21Qwen3 14B Base· Open Arabic LLM Leaderboard data47.8%
22Qwen3 1.7B Base· Open Arabic LLM Leaderboard data43.9%
23Llama 3.1 8B· Open Arabic LLM Leaderboard data42.7%
24Phi 4 Mini· Open Arabic LLM Leaderboard data42.2%
25Qwen2.5 7B Instruct· Open Arabic LLM Leaderboard data40.3%
26Phi 4· Open Arabic LLM Leaderboard data39.1%
27Qwen1.5-14B· Open Arabic LLM Leaderboard data38%
28Qwen3 30B A3B Instruct· Open Arabic LLM Leaderboard data37.4%
29Qwen2 1.5b Instruct· Open Arabic LLM Leaderboard data37.4%
30Qwen2.5 3B Instruct· Open Arabic LLM Leaderboard data35.9%
31Qwen2.5 1.5B Instruct· Open Arabic LLM Leaderboard data35.9%
32Qwen1.5-7B· Open Arabic LLM Leaderboard data34.4%
33Qwen3 14B· Open Arabic LLM Leaderboard data29.9%
34Qwen1.5-4B· Open Arabic LLM Leaderboard data29.8%
35qwen2-0.5b-instruct· Open Arabic LLM Leaderboard data29.6%
36Qwen3 32B· Open Arabic LLM Leaderboard data29.5%
37Qwen2.5 0.5b Instruct· Open Arabic LLM Leaderboard data27.9%
38Qwen3 8B· Open Arabic LLM Leaderboard data27.3%
39Qwen3 4B· Open Arabic LLM Leaderboard data26.6%
40Llama 3.2 3B· Open Arabic LLM Leaderboard data26%
41Qwen2.5 1.5B· Open Arabic LLM Leaderboard data25.7%
42Qwen3 0.6B· Open Arabic LLM Leaderboard data25.5%
43Gemma 2 9B IT· Open Arabic LLM Leaderboard data25%
44DeepSeek R1 Distill Qwen 14B· Open Arabic LLM Leaderboard data24.1%
45Llama 3.2 1B· Open Arabic LLM Leaderboard data23.9%
46Qwen1.5-1.8B· Open Arabic LLM Leaderboard data23.6%
47Qwen3 4B Instruct 2507· Open Arabic LLM Leaderboard data23.5%
48Gemma 2 2B IT· Open Arabic LLM Leaderboard data23.4%
49QwQ 32B· Open Arabic LLM Leaderboard data23.4%
50DeepSeek R1 Distill Qwen 32B· Open Arabic LLM Leaderboard data23.4%

Showing 1–50 of 51 results.

Data credits: Open Arabic LLM Leaderboard (OALL), Apache-2.0 (licence)