Skip to content

Benchmark

OALL MadinahQA

Scores are percentages.

Models measured
51
Scores recorded
51
Sources
1
IQ weight
0.55

Score distribution

42.9%
Mean
40%
Median
14.1%
Std dev
25.3%
10th percentile
61.4%
90th percentile
20.1%
Min
20.1%72.9%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.55 for chat models, measured over 51 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Llama 3.3 70B Instruct· Open Arabic LLM Leaderboard data72.9%
2Gemma 3 27B IT· Open Arabic LLM Leaderboard data69.4%
3Qwen2.5 72B· Open Arabic LLM Leaderboard data65%
4C4AI Command R7b Arabic 02 2025· Open Arabic LLM Leaderboard data63.8%
5Qwen2.5 7B Instruct· Open Arabic LLM Leaderboard data62.7%
6Qwen 2 (72B)· Open Arabic LLM Leaderboard data61.4%
7Qwen2.5 14B· Open Arabic LLM Leaderboard data59.9%
8Gemma 3 27B Pt· Open Arabic LLM Leaderboard data59.5%
9Qwen2 7B Instruct· Open Arabic LLM Leaderboard data59.5%
10Qwen2.5 72B Instruct· Open Arabic LLM Leaderboard data58.1%
11Qwen2.5 32B Instruct· Open Arabic LLM Leaderboard data57.1%
12Qwen 2.5 32B Abliterated· Open Arabic LLM Leaderboard data56.5%
13Qwen2.5 32B· Open Arabic LLM Leaderboard data55.8%
14Qwen3 4B Base· Open Arabic LLM Leaderboard data54%
15Aya Expanse 32B· Open Arabic LLM Leaderboard data53.5%
16Qwen3 14B Base· Open Arabic LLM Leaderboard data52.8%
17Qwen3 8B Base· Open Arabic LLM Leaderboard data52.2%
18Qwen1.5-14B· Open Arabic LLM Leaderboard data51.4%
19Qwen2.5 7B· Open Arabic LLM Leaderboard data51.1%
20Aya Expanse 8B· Open Arabic LLM Leaderboard data48.7%
21Phi 4 Mini· Open Arabic LLM Leaderboard data47.1%
22Qwen2.5 3B Instruct· Open Arabic LLM Leaderboard data47%
23Qwen2 1.5b Instruct· Open Arabic LLM Leaderboard data45.5%
24Llama 3.1 8B· Open Arabic LLM Leaderboard data43.1%
25Qwen1.5-7B· Open Arabic LLM Leaderboard data40.2%
26Qwen3 1.7B Base· Open Arabic LLM Leaderboard data40%
27Llama 3.1 8B Instruct· Open Arabic LLM Leaderboard data39.7%
28Phi 4· Open Arabic LLM Leaderboard data39.2%
29Llama 3.2 3B· Open Arabic LLM Leaderboard data38.7%
30Qwen2.5 1.5B Instruct· Open Arabic LLM Leaderboard data38.2%
31Qwen 2.5 14B Instruct· Open Arabic LLM Leaderboard data35.6%
32Qwen3 0.6B· Open Arabic LLM Leaderboard data35.4%
33Qwen3 14B· Open Arabic LLM Leaderboard data35.2%
34Gemma 2 27B IT· Open Arabic LLM Leaderboard data33.3%
35Qwen3 32B· Open Arabic LLM Leaderboard data33%
36Qwen1.5-4B· Open Arabic LLM Leaderboard data31.8%
37Gemma 2 9B IT· Open Arabic LLM Leaderboard data31.7%
38DeepSeek R1 Distill Qwen 32B· Open Arabic LLM Leaderboard data29.4%
39gemma2-27b· Open Arabic LLM Leaderboard data29.4%
40QwQ 32B· Open Arabic LLM Leaderboard data29.4%
41DeepSeek R1 Distill Qwen 14B· Open Arabic LLM Leaderboard data29.1%
42qwen2-0.5b-instruct· Open Arabic LLM Leaderboard data28.2%
43Qwen3 4B· Open Arabic LLM Leaderboard data27.2%
44Qwen1.5-1.8B· Open Arabic LLM Leaderboard data26%
45Llama 3.2 1B· Open Arabic LLM Leaderboard data25.7%
46Qwen3 8B· Open Arabic LLM Leaderboard data25.3%
47Qwen3 30B A3B Instruct· Open Arabic LLM Leaderboard data25.1%
48Qwen3 4B Instruct 2507· Open Arabic LLM Leaderboard data25.1%
49Qwen2.5 0.5b Instruct· Open Arabic LLM Leaderboard data24.8%
50Qwen2.5 1.5B· Open Arabic LLM Leaderboard data23.1%

Showing 1–50 of 51 results.

Data credits: Open Arabic LLM Leaderboard (OALL), Apache-2.0 (licence)