Benchmark
Benchmark Nusantara id-B pilihan-ganda (prompt mcq-py1, scorer mcq-py1, items 3e5973f02ae8, key 4e94e1cb7f25, withheld slice)
Multilingual benchmarks compare model performance across languages. Higher scores are better.
- Models measured
- 9
- Scores recorded
- 9
- Sources
- 1
- IQ weight
- 0.74
Score distribution
- 0.889
- Mean
- 0.909
- Median
- 0.094
- Std dev
- 0.727
- 10th percentile
- 1
- 90th percentile
- 0.727
- Min
This benchmark has a weight of 0.74 for chat models, measured over 9 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.
Leaderboard
| # | Model | Score |
|---|---|---|
| 1 | Claude Haiku 4.5repetition: 1, route anthropic/claude-haiku-4.5, token cap 600· Benchmark Nusantara (ibahasa) data docs | 1 |
| 2 | GPT-5.6 Lunarepetition: 1, route openai/gpt-5.6-luna, token cap 600· Benchmark Nusantara (ibahasa) data docs | 1 |
| 3 | DeepSeek V4 Flash 0731repetition: 1, route deepseek/deepseek-v4-flash-0731, token cap 2400· Benchmark Nusantara (ibahasa) data docs | 0.909 |
| 4 | Gemma 4 31B ITrepetition: 1, route google/gemma-4-31b-it, token cap 600· Benchmark Nusantara (ibahasa) data docs | 0.909 |
| 5 | Ling 3.0 Flashrepetition: 1, route inclusionai/ling-3.0-flash, token cap 2400· Benchmark Nusantara (ibahasa) data docs | 0.909 |
| 6 | Mistral Small 24B Instructrepetition: 1, route mistralai/mistral-small-24b-instruct-2501, token cap 600, 2501· Benchmark Nusantara (ibahasa) data docs | 0.909 |
| 7 | Qwen3 30B A3B Instructrepetition: 1, route qwen/qwen3-30b-a3b-instruct-2507, token cap 600· Benchmark Nusantara (ibahasa) data docs | 0.909 |
| 8 | Llama 3.1 8B Instructrepetition: 1, local Q4_K_M, token cap 600· Benchmark Nusantara (ibahasa) data docs | 0.727 |
| 9 | Solar Pro 4repetition: 1, route upstage/solar-pro4, token cap 600· Benchmark Nusantara (ibahasa) data docs | 0.727 |
Data credits: Benchmark Nusantara, ibahasa (M. Khoirul Huda), doi:10.5281/zenodo.21970463. Test results CC BY 4.0, attribution to ibahasa; aggregate results CSV only. (licence)