Skip to content

Benchmark

Benchmark Nusantara id-B pilihan-ganda (prompt mcq-py1, scorer mcq-py1, items d3e8c01bc554, key 4e94e1cb7f25, withheld slice)

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models measured
17
Scores recorded
17
Sources
1
IQ weight
1.10

Score distribution

0.908
Mean
0.944
Median
0.081
Std dev
0.833
10th percentile
1
90th percentile
0.722
Min
0.7221
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.10 for chat models, measured over 17 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Haiku 4.5repetition: 1, route anthropic/claude-haiku-4.5, token cap 600· Benchmark Nusantara (ibahasa) data docs1
2DeepSeek V4 Flash 0731repetition: 1, route deepseek/deepseek-v4-flash-0731, token cap 2400· Benchmark Nusantara (ibahasa) data docs1
3GPT-5.6 Lunarepetition: 1, route openai/gpt-5.6-luna, token cap 2400· Benchmark Nusantara (ibahasa) data docs1
4Claude Haiku 5.5repetition: 1, route anthropic/claude-haiku-5.5, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.944
5Claude Sonnet 5.5repetition: 1, route anthropic/claude-sonnet-5.5, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.944
6DeepSeek V4.1 Flashrepetition: 1, route deepseek/deepseek-v4.1-flash, token cap 9600· Benchmark Nusantara (ibahasa) data docs0.944
7Gemma 4 31B ITrepetition: 1, route google/gemma-4-31b-it, token cap 600· Benchmark Nusantara (ibahasa) data docs0.944
8GPT-6 Astrarepetition: 1, route openai/gpt-6-astra, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.944
9Ling 3.0 Flashrepetition: 1, route inclusionai/ling-3.0-flash, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.944
10MiMo V2.6 Flashrepetition: 1, route xiaomi/mimo-v2.6-flash, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.944
11GLM 5.3repetition: 1, route z-ai/glm-5.3, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.889
12GLM 5.3 Flashrepetition: 1, route z-ai/glm-5.3-flash, token cap 9600· Benchmark Nusantara (ibahasa) data docs0.889
13Qwen3 30B A3B Instructrepetition: 1, route qwen/qwen3-30b-a3b-instruct-2507, token cap 600· Benchmark Nusantara (ibahasa) data docs0.889
14Seed 2.0 Minirepetition: 1, route bytedance-seed/seed-2.0-mini, token cap 8000· Benchmark Nusantara (ibahasa) data docs0.889
15Solar Pro 4repetition: 1, route upstage/solar-pro4, token cap 600· Benchmark Nusantara (ibahasa) data docs0.833
16Llama 3.1 8B Instructrepetition: 1, local Q4_K_M, token cap 600· Benchmark Nusantara (ibahasa) data docs0.722
17Mistral Small 24B Instructrepetition: 1, route mistralai/mistral-small-24b-instruct-2501, token cap 600, 2501· Benchmark Nusantara (ibahasa) data docs0.722

Data credits: Benchmark Nusantara, ibahasa (M. Khoirul Huda), doi:10.5281/zenodo.21970463. Test results CC BY 4.0, attribution to ibahasa; aggregate results CSV only. (licence)