Skip to content

Benchmark

Benchmark Nusantara id-B pilihan-ganda (prompt mcq-py1, scorer mcq-py1, items 3e5973f02ae8, key 4e94e1cb7f25, withheld slice)

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models measured
9
Scores recorded
9
Sources
1
IQ weight
0.74

Score distribution

0.889
Mean
0.909
Median
0.094
Std dev
0.727
10th percentile
1
90th percentile
0.727
Min

This benchmark has a weight of 0.74 for chat models, measured over 9 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Haiku 4.5repetition: 1, route anthropic/claude-haiku-4.5, token cap 600· Benchmark Nusantara (ibahasa) data docs1
2GPT-5.6 Lunarepetition: 1, route openai/gpt-5.6-luna, token cap 600· Benchmark Nusantara (ibahasa) data docs1
3DeepSeek V4 Flash 0731repetition: 1, route deepseek/deepseek-v4-flash-0731, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.909
4Gemma 4 31B ITrepetition: 1, route google/gemma-4-31b-it, token cap 600· Benchmark Nusantara (ibahasa) data docs0.909
5Ling 3.0 Flashrepetition: 1, route inclusionai/ling-3.0-flash, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.909
6Mistral Small 24B Instructrepetition: 1, route mistralai/mistral-small-24b-instruct-2501, token cap 600, 2501· Benchmark Nusantara (ibahasa) data docs0.909
7Qwen3 30B A3B Instructrepetition: 1, route qwen/qwen3-30b-a3b-instruct-2507, token cap 600· Benchmark Nusantara (ibahasa) data docs0.909
8Llama 3.1 8B Instructrepetition: 1, local Q4_K_M, token cap 600· Benchmark Nusantara (ibahasa) data docs0.727
9Solar Pro 4repetition: 1, route upstage/solar-pro4, token cap 600· Benchmark Nusantara (ibahasa) data docs0.727

Data credits: Benchmark Nusantara, ibahasa (M. Khoirul Huda), doi:10.5281/zenodo.21970463. Test results CC BY 4.0, attribution to ibahasa; aggregate results CSV only. (licence)