Skip to content

Benchmark

Benchmark Nusantara id-N terbuka (prompt open-py3, scorer open-py1, items 5ca9dfe11dc2, key eda1d1cddc75, public slice)

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models measured
13
Scores recorded
13
Sources
1
IQ weight
1.74

Score distribution

0.659
Mean
0.694
Median
0.158
Std dev
0.469
10th percentile
0.837
90th percentile
0.245
Min
0.2450.878
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.74 for chat models, measured over 13 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemma 4 31B ITrepetition: 1, route google/gemma-4-31b-it, token cap 600· Benchmark Nusantara (ibahasa) data docs0.878
2GPT-5.6 Lunarepetition: 1, route openai/gpt-5.6-luna, token cap 600· Benchmark Nusantara (ibahasa) data docs0.837
3DeepSeek V4 Flash 0731repetition: 1, route deepseek/deepseek-v4-flash-0731, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.755
4GLM 5.3repetition: 1, route z-ai/glm-5.3, token cap 6000· Benchmark Nusantara (ibahasa) data docs0.755
5Claude Haiku 5.5repetition: 1, route anthropic/claude-haiku-5.5, token cap 6000· Benchmark Nusantara (ibahasa) data docs0.735
6Seed 2.0 Minirepetition: 1, route bytedance-seed/seed-2.0-mini, token cap 4000· Benchmark Nusantara (ibahasa) data docs0.714
7GLM 5.3 Flashrepetition: 1, route z-ai/glm-5.3-flash, token cap 6000· Benchmark Nusantara (ibahasa) data docs0.694
8Claude Sonnet 5.5repetition: 1, route anthropic/claude-sonnet-5.5, token cap 6000· Benchmark Nusantara (ibahasa) data docs0.653
9Qwen3 30B A3B Instructrepetition: 1, route qwen/qwen3-30b-a3b-instruct-2507, token cap 600· Benchmark Nusantara (ibahasa) data docs0.633
10Solar Pro 4repetition: 1, route upstage/solar-pro4, token cap 600· Benchmark Nusantara (ibahasa) data docs0.612
11Ling 3.0 Flashrepetition: 1, route inclusionai/ling-3.0-flash, token cap 24000· Benchmark Nusantara (ibahasa) data docs0.592
12Mistral Small 24B Instructrepetition: 1, route mistralai/mistral-small-24b-instruct-2501, token cap 600, 2501· Benchmark Nusantara (ibahasa) data docs0.469
13Llama 3.1 8B Instructrepetition: 1, local Q4_K_M, token cap 600· Benchmark Nusantara (ibahasa) data docs0.245

Data credits: Benchmark Nusantara, ibahasa (M. Khoirul Huda), doi:10.5281/zenodo.21970463. Test results CC BY 4.0, attribution to ibahasa; aggregate results CSV only. (licence)