Skip to content

Benchmark

Benchmark Nusantara id-W terbuka (prompt open-py1, scorer open-py1, items 27af2c184856, key 82bbf3cc2f8a, public slice)

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models measured
15
Scores recorded
15
Sources
1
IQ weight
1.11

Score distribution

0.63
Mean
0.658
Median
0.072
Std dev
0.526
10th percentile
0.737
90th percentile
0.5
Min
0.50.763
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.11 for chat models, measured over 15 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemma 4 31B ITrepetition: 1, route google/gemma-4-31b-it, token cap 600· Benchmark Nusantara (ibahasa) data docs0.763
2Claude Sonnet 5.5repetition: 1, route anthropic/claude-sonnet-5.5, token cap 4000· Benchmark Nusantara (ibahasa) data docs0.737
3GLM 5.3repetition: 1, route z-ai/glm-5.3, token cap 4000· Benchmark Nusantara (ibahasa) data docs0.684
4MiMo V2.6 Flashrepetition: 1, route xiaomi/mimo-v2.6-flash, token cap 4000· Benchmark Nusantara (ibahasa) data docs0.684
5Claude Haiku 5.5repetition: 1, route anthropic/claude-haiku-5.5, token cap 4000· Benchmark Nusantara (ibahasa) data docs0.658
6GLM 5.3 Flashrepetition: 1, route z-ai/glm-5.3-flash, token cap 4000· Benchmark Nusantara (ibahasa) data docs0.658
7GPT-5.6 Lunarepetition: 1, route openai/gpt-5.6-luna, token cap 600· Benchmark Nusantara (ibahasa) data docs0.658
8Ling 3.0 Flashrepetition: 1, route inclusionai/ling-3.0-flash, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.658
9DeepSeek V4 Flash 0731repetition: 1, route deepseek/deepseek-v4-flash-0731, token cap 9600· Benchmark Nusantara (ibahasa) data docs0.605
10Solar Pro 4repetition: 1, route upstage/solar-pro4, token cap 600· Benchmark Nusantara (ibahasa) data docs0.605
11Mistral Small 24B Instructrepetition: 1, route mistralai/mistral-small-24b-instruct-2501, token cap 600, 2501· Benchmark Nusantara (ibahasa) data docs0.579
12Seed 2.0 Minirepetition: 1, route bytedance-seed/seed-2.0-mini, token cap 4000· Benchmark Nusantara (ibahasa) data docs0.579
13Qwen3 30B A3B Instructrepetition: 1, route qwen/qwen3-30b-a3b-instruct-2507, token cap 600· Benchmark Nusantara (ibahasa) data docs0.553
14Claude Haiku 4.5repetition: 1, route anthropic/claude-haiku-4.5, token cap 600· Benchmark Nusantara (ibahasa) data docs0.526
15Llama 3.1 8B Instructrepetition: 1, local Q4_K_M, token cap 600· Benchmark Nusantara (ibahasa) data docs0.5

Data credits: Benchmark Nusantara, ibahasa (M. Khoirul Huda), doi:10.5281/zenodo.21970463. Test results CC BY 4.0, attribution to ibahasa; aggregate results CSV only. (licence)