Skip to content

Benchmark

Benchmark Nusantara id-N terbuka (prompt open-py1, scorer open-py1, items 5ca9dfe11dc2, key eda1d1cddc75, withheld slice)

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models measured
14
Scores recorded
14
Sources
1
IQ weight
0.00

Score distribution

0.518
Mean
0.55
Median
0.075
Std dev
0.45
10th percentile
0.6
90th percentile
0.35
Min
0.350.65
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.00 for chat models, measured over 14 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Mistral Small 24B Instructrepetition: 1, route mistralai/mistral-small-24b-instruct-2501, token cap 600, 2501· Benchmark Nusantara (ibahasa) data docs0.65
2Ling 3.0 Flashrepetition: 1, route inclusionai/ling-3.0-flash, token cap 4000· Benchmark Nusantara (ibahasa) data docs0.6
3MiMo V2.6 Flashrepetition: 1, route xiaomi/mimo-v2.6-flash, token cap 6000· Benchmark Nusantara (ibahasa) data docs0.6
4DeepSeek V4 Flash 0731repetition: 1, route deepseek/deepseek-v4-flash-0731, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.55
5Gemma 4 31B ITrepetition: 1, route google/gemma-4-31b-it, token cap 600· Benchmark Nusantara (ibahasa) data docs0.55
6GLM 5.3repetition: 1, route z-ai/glm-5.3, token cap 6000· Benchmark Nusantara (ibahasa) data docs0.55
7Qwen3 30B A3B Instructrepetition: 1, route qwen/qwen3-30b-a3b-instruct-2507, token cap 600· Benchmark Nusantara (ibahasa) data docs0.55
8GLM 5.3 Flashrepetition: 1, route z-ai/glm-5.3-flash, token cap 6000· Benchmark Nusantara (ibahasa) data docs0.5
9Llama 3.1 8B Instructrepetition: 1, local Q4_K_M, token cap 600· Benchmark Nusantara (ibahasa) data docs0.5
10Seed 2.0 Minirepetition: 1, route bytedance-seed/seed-2.0-mini, token cap 4000· Benchmark Nusantara (ibahasa) data docs0.5
11Claude Haiku 5.5repetition: 1, route anthropic/claude-haiku-5.5, token cap 6000· Benchmark Nusantara (ibahasa) data docs0.45
12GPT-5.6 Lunarepetition: 1, route openai/gpt-5.6-luna, token cap 600· Benchmark Nusantara (ibahasa) data docs0.45
13Solar Pro 4repetition: 1, route upstage/solar-pro4, token cap 600· Benchmark Nusantara (ibahasa) data docs0.45
14Claude Sonnet 5.5repetition: 1, route anthropic/claude-sonnet-5.5, token cap 6000· Benchmark Nusantara (ibahasa) data docs0.35

Data credits: Benchmark Nusantara, ibahasa (M. Khoirul Huda), doi:10.5281/zenodo.21970463. Test results CC BY 4.0, attribution to ibahasa; aggregate results CSV only. (licence)