Skip to content

Benchmark

Arena ELO - Japanese

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
245
Scores recorded
286
Sources
1
IQ weight
1.65

Score distribution

1,296.5
Mean
1,336.3
Median
150
Std dev
1,061.5
10th percentile
1,461.1
90th percentile
878.3
Min
878.31,557.4
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.65 for chat models, measured over 245 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
251Qwen2 72B Instruct· LMArena data1,111.2
252Mistral Small 24B Instruct2501· LMArena data1,111.1
253Claude 3 Haiku20240307· LMArena data1,102.6
254Gemma 2 2B IT· LMArena data1,083.3
255Reka Flash 21BOnline, 20240226· LMArena data1,075.2
256Qwen1.5-110B-Chat· LMArena data1,074.3
257Qwen1.5-72B-Chat· LMArena data1,061.5
258Llama 3.1 8B Instruct· LMArena data1,061.2
259Reka Flash 21B20240226· LMArena data1,049.1
260GPT-3.5 Turbo0125· LMArena data1,043.6
261Phi-3 Medium 4K Instruct· LMArena data1,042.6
262Mixtral 8x22B Instruct v0.1· LMArena data1,036.9
263Qwen1.5-32B-Chat· LMArena data1,027.4
264Gemini ProDev API· LMArena data1,022.8
265Yi 1.5 34B Chat· LMArena data1,021.6
266Qwen1.5-14B-Chat· LMArena data1,019.5
267Jamba 1.5 Mini· LMArena data1,019.3
268Llama 3 70B Instruct· LMArena data1,016.8
269Mistral Medium· LMArena data1,012.7
270Mistral Large2402· LMArena data1,008.4
271Starling LM 7B Beta· LMArena data1,000.8
272Yi 34B Chat· LMArena data993.3
273DBRX Instruct Preview· LMArena data989.8
274Gemma 1.1 7B IT· LMArena data970.9
275Snowflake Arctic Instruct· LMArena data969.7
276Llama 3 8B Instruct· LMArena data967.7
277Phi-3 Small 8K Instruct· LMArena data965.9
278Vicuna 33B· LMArena data958.8
279Phi-3 Mini 4K InstructJune 2024· LMArena data954.6
280Phi-3 Mini 4K Instruct· LMArena data935.2
281Mixtral 8x7B Instruct v0.1· LMArena data931.4
282Llama 2 70B Chat· LMArena data927
283Vicuna 13B· LMArena data918.6
284Phi-3 Mini 128K Instruct· LMArena data899.4
285Llama 2 13B Chat· LMArena data894.2
286Mistral 7B Instruct v0.2· LMArena data878.3

Showing 251–286 of 286 results.

Data credits: LMArena, CC BY 4.0 (licence)