Skip to content

Benchmark

Benchmark Nusantara id-B pilihan-ganda (prompt mcq-py1, scorer mcq-py1, items d3e8c01bc554, key 4e94e1cb7f25, public slice)

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models measured
17
Scores recorded
17
Sources
1
IQ weight
1.34

Score distribution

0.915
Mean
0.929
Median
0.068
Std dev
0.881
10th percentile
0.976
90th percentile
0.762
Min
0.7621
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.34 for chat models, measured over 17 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT-5.6 Lunarepetition: 1, route openai/gpt-5.6-luna, token cap 2400· Benchmark Nusantara (ibahasa) data docs1
2GPT-6 Astrarepetition: 1, route openai/gpt-6-astra, token cap 2400· Benchmark Nusantara (ibahasa) data docs1
3DeepSeek V4.1 Flashrepetition: 1, route deepseek/deepseek-v4.1-flash, token cap 9600· Benchmark Nusantara (ibahasa) data docs0.976
4Gemma 4 31B ITrepetition: 1, route google/gemma-4-31b-it, token cap 600· Benchmark Nusantara (ibahasa) data docs0.976
5Claude Haiku 5.5repetition: 1, route anthropic/claude-haiku-5.5, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.952
6Claude Sonnet 5.5repetition: 1, route anthropic/claude-sonnet-5.5, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.952
7GLM 5.3repetition: 1, route z-ai/glm-5.3, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.952
8MiMo V2.6 Flashrepetition: 1, route xiaomi/mimo-v2.6-flash, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.952
9DeepSeek V4 Flash 0731repetition: 1, route deepseek/deepseek-v4-flash-0731, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.929
10Claude Haiku 4.5repetition: 1, route anthropic/claude-haiku-4.5, token cap 600· Benchmark Nusantara (ibahasa) data docs0.905
11GLM 5.3 Flashrepetition: 1, route z-ai/glm-5.3-flash, token cap 9600· Benchmark Nusantara (ibahasa) data docs0.905
12Ling 3.0 Flashrepetition: 1, route inclusionai/ling-3.0-flash, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.881
13Qwen3 30B A3B Instructrepetition: 1, route qwen/qwen3-30b-a3b-instruct-2507, token cap 600· Benchmark Nusantara (ibahasa) data docs0.881
14Seed 2.0 Minirepetition: 1, route bytedance-seed/seed-2.0-mini, token cap 8000· Benchmark Nusantara (ibahasa) data docs0.881
15Solar Pro 4repetition: 1, route upstage/solar-pro4, token cap 600· Benchmark Nusantara (ibahasa) data docs0.881
16Llama 3.1 8B Instructrepetition: 1, local Q4_K_M, token cap 600· Benchmark Nusantara (ibahasa) data docs0.762
17Mistral Small 24B Instructrepetition: 1, route mistralai/mistral-small-24b-instruct-2501, token cap 600, 2501· Benchmark Nusantara (ibahasa) data docs0.762

Data credits: Benchmark Nusantara, ibahasa (M. Khoirul Huda), doi:10.5281/zenodo.21970463. Test results CC BY 4.0, attribution to ibahasa; aggregate results CSV only. (licence)