Skip to content

Benchmark

SeaLLMs SeaBench private (Indonesian)

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models measured
32
Scores recorded
32
Sources
1
IQ weight
2.51

Score distribution

7.8
Mean
8.2
Median
1.1
Std dev
6.1
10th percentile
8.8
90th percentile
4.5
Min
4.59
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.51 for chat models, measured over 32 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude 3.5 Sonnetjudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs9
2GPT 4.1judge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs8.9
3Claude Sonnet 3.7judge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs8.9
4DeepSeek V3judge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs8.8
5gemini-pro-1.5judge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs8.8
6GPT-4.1 minijudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs8.8
7GPT 4ojudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs8.7
8Gemini 1.5 Flashjudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs8.6
9Gemini 2.0 Flashjudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs8.5
10Gemma 3 27B ITjudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs8.5
11Claude Haiku 3.5judge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs8.4
12Gemini 2.0 Flash Litejudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs8.3
13GPT-4o minijudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs8.3
14Gemma 2 27B ITjudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs8.2
15Llama 4 Maverickjudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs8.2
16Gemma 3 12B ITjudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs8.2
17GPT-4.1 nanojudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs8.1
18Qwen2.5 72B Instructjudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs8.1
19Gemma 2 9B ITjudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs8
20Qwen2 72B Instructjudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs8
21Claude 3 Haikujudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs7.8
22Qwen 2.5 14B Instructjudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs7.7
23Llama 3 70B Instructjudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs7.7
24Gemma 3 4B ITjudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs7.4
25Llama 3.1 70B Instructjudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs7.4
26Llama 4 Scoutjudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs7.1
27Qwen2.5 7B Instructjudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs6.9
28Llama 3.1 8B Instructjudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs6.4
29Qwen2 7B Instructjudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs6.1
30GLM-4-9b-chatjudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs6
31Qwen2.5 3B Instructjudge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs5.6
32Mistral (7B) Instruct v0.3judge:GPT-4o-2024-08-06· SeaLLMs Leaderboard for SEA Languages data docs4.5

Data credits: SeaExam and SeaBench leaderboard, SeaLLMs (DAMO-NLP-SG). Published leaderboard scores; results licence not stated.