Skip to content

Benchmark

HELM Lite: WMT 2014 - BLEU-4

Multilingual benchmarks compare model performance across languages. Scores are percentages.

Models measured
69
Scores recorded
70
Sources
1
IQ weight
1.81

Score distribution

18.5%
Mean
19.2%
Median
4.2%
Std dev
14.2%
10th percentile
22.5%
90th percentile
2.3%
Min
2.3%26%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.81 for chat models, measured over 69 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1text-unicorn· Stanford HELM Lite data26%
2Claude 3 Opus· Stanford HELM Lite data24%
3Llama 3.1 405B Instruct Turbo· Stanford HELM Lite data23.8%
4gemini-1.5-pro-002· Stanford HELM Lite data23.1%
5GPT 4o· Stanford HELM Lite data23.1%
6Claude 3.5 Sonnet· Stanford HELM Lite data22.9%
7Nova Prov1· Stanford HELM Lite data22.9%
8gemini-1.5-flash-001· Stanford HELM Lite data22.5%
9Meta Llama 3 70b· Stanford HELM Lite data22.5%
10Llama-3.2-90B-Vision-Instruct-Turbo· Stanford HELM Lite data22.4%
11Meta Llama 3.1 70B Instruct Turbo· Stanford HELM Lite data22.3%
12Llama 3.3 70B Instruct Turbo· Stanford HELM Lite data21.9%
13Claude 2· Stanford HELM Lite data21.9%
14Claude 3 Sonnet· Stanford HELM Lite data21.8%
15GPT-4 Turbo· Stanford HELM Lite data21.8%
16Gemma 2 27B IT· Stanford HELM Lite data21.4%
17gemini-1.5-flash-002· Stanford HELM Lite data21.2%
18Gemini 2.0 Flash Experimental· Stanford HELM Lite data21.2%
19GPT 4· Stanford HELM Lite data21.1%
20DeepSeek V3· Stanford HELM Lite data20.9%
21mixtral-8x22b· Stanford HELM Lite data20.9%
22Qwen2 72B Instruct· Stanford HELM Lite data20.7%
23Qwen2.5 72B Instruct Turbo· Stanford HELM Lite data20.7%
24GPT-4o mini· Stanford HELM Lite data20.6%
25GPT 4 1106 Preview· Stanford HELM Lite data20.5%
26Claude 2.1· Stanford HELM Lite data20.4%
27Nova Litev1· Stanford HELM Lite data20.4%
28Command R Plus· Stanford HELM Lite data20.3%
29Jamba 1.5 Large· Stanford HELM Lite data20.3%
30Qwen1.5-72B· Stanford HELM Lite data20.1%
31Gemma 2 9B IT· Stanford HELM Lite data20.1%
32Llama 2 70b· Stanford HELM Lite data19.6%
33Claude Instant 1.2· Stanford HELM Lite data19.4%
34Mistral Large2407· Stanford HELM Lite data19.2%
35Qwen1.5-110B-Chat· Stanford HELM Lite data19.2%
36Nova Microv1· Stanford HELM Lite data19.2%
37Text Davinci 003· Stanford HELM Lite data19.1%
38gemini-1.5-pro-001· Stanford HELM Lite data18.9%
39GPT-3.5 Turbo· Stanford HELM Lite data18.7%
40gemma-7b· Stanford HELM Lite data18.7%
41DeepSeek LLM 67B Chat· Stanford HELM Lite data18.6%
42Meta Llama 3 8b· Stanford HELM Lite data18.3%
43Mistral Large· Stanford HELM Lite data18.2%
44Llama 3.1 8B Instruct Turbo· Stanford HELM Lite data18.1%
45Llama-3.2-11B-Vision-Instruct-Turbo· Stanford HELM Lite data17.9%
46Jamba 1.5 Mini· Stanford HELM Lite data17.9%
47Qwen1.5-14B· Stanford HELM Lite data17.8%
48Open Mistral Nemo· Stanford HELM Lite data17.7%
49Text Davinci 002· Stanford HELM Lite data17.4%
50Snowflake Arctic Instruct· Stanford HELM Lite data17.2%

Showing 1–50 of 70 results.

Data credits: Stanford HELM (Stanford CRFM), Apache-2.0 (licence)