Skip to content

Benchmark

Benchmark Nusantara jv-A pilihan-ganda (prompt mcq-py1, scorer mcq-py1, items c0c83850858e, key 18e8ec378904, public slice)

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models measured
18
Scores recorded
18
Sources
1
IQ weight
1.31

Score distribution

0.757
Mean
0.815
Median
0.203
Std dev
0.482
10th percentile
0.963
90th percentile
0.333
Min
0.3330.963
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.31 for chat models, measured over 18 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Sonnet 5.5repetition: 1, route anthropic/claude-sonnet-5.5, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.963
2DeepSeek V4.1 Flashrepetition: 1, route deepseek/deepseek-v4.1-flash, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.963
3Gemini 3.5 Flashrepetition: 1, route google/gemini-3.5-flash, token cap 600· Benchmark Nusantara (ibahasa) data docs0.963
4Gemma 4 31B ITrepetition: 1, route google/gemma-4-31b-it, token cap 600· Benchmark Nusantara (ibahasa) data docs0.963
5GPT-6 Astrarepetition: 1, route openai/gpt-6-astra, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.963
6DeepSeek V4 Flash 0731repetition: 1, route deepseek/deepseek-v4-flash-0731, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.926
7Claude Haiku 5.5repetition: 1, route anthropic/claude-haiku-5.5, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.889
8GPT-5.6 Lunarepetition: 1, route openai/gpt-5.6-luna, token cap 600· Benchmark Nusantara (ibahasa) data docs0.852
9GLM 5.3 Flashrepetition: 1, route z-ai/glm-5.3-flash, token cap 9600· Benchmark Nusantara (ibahasa) data docs0.815
10MiMo V2.6 Flashrepetition: 1, route xiaomi/mimo-v2.6-flash, token cap 9600· Benchmark Nusantara (ibahasa) data docs0.815
11GLM 5.3repetition: 1, route z-ai/glm-5.3, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.778
12Claude Haiku 4.5repetition: 1, route anthropic/claude-haiku-4.5, token cap 600· Benchmark Nusantara (ibahasa) data docs0.741
13Ling 3.0 Flashrepetition: 1, route inclusionai/ling-3.0-flash, token cap 2400· Benchmark Nusantara (ibahasa) data docs0.667
14Solar Pro 4repetition: 1, route upstage/solar-pro4, token cap 600· Benchmark Nusantara (ibahasa) data docs0.593
15Qwen3 30B A3B Instructrepetition: 1, route qwen/qwen3-30b-a3b-instruct-2507, token cap 600· Benchmark Nusantara (ibahasa) data docs0.519
16Seed 2.0 Minirepetition: 1, route bytedance-seed/seed-2.0-mini, token cap 12000· Benchmark Nusantara (ibahasa) data docs0.482
17Mistral Small 24B Instructrepetition: 1, route mistralai/mistral-small-24b-instruct-2501, token cap 600, 2501· Benchmark Nusantara (ibahasa) data docs0.407
18Llama 3.1 8B Instructrepetition: 1, local Q4_K_M, token cap 600· Benchmark Nusantara (ibahasa) data docs0.333

Data credits: Benchmark Nusantara, ibahasa (M. Khoirul Huda), doi:10.5281/zenodo.21970463. Test results CC BY 4.0, attribution to ibahasa; aggregate results CSV only. (licence)