Skip to content

Benchmark

NYT Connections – Older models (Lech Mazur)

Scores are percentages.

Models measured
77
Scores recorded
93
Sources
1
IQ weight
3.37

Score distribution

27.2%
Mean
13.9%
Median
26%
Std dev
4.2%
10th percentile
70.5%
90th percentile
0.7%
Min
0.7%89.5%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 3.37 for chat models, measured over 77 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
51Command A· Lech Mazur suite data13.3%
52Gemini 2.5 Flash· Lech Mazur suite data12.8%
53GLM 4.6· Lech Mazur suite data12.5%
54Grok 4 Fastnon-reasoning· Lech Mazur suite data12.5%
55Gemini 2.0 Flash Experimentalthink· Lech Mazur suite data12%
56GPT 4o· Lech Mazur suite data11.8%
57Qwen3 Max Preview· Lech Mazur suite data11.6%
58gemini-2.0-pro-exp-02-05· Lech Mazur suite data11.2%
59Kimi K2 0905· Lech Mazur suite data11%
60GPT 4.1· Lech Mazur suite data10.8%
61MiniMax M1· Lech Mazur suite data10.5%
62GPT 5.1no reasoning· Lech Mazur suite data10%
63Grok 3no reasoning, beta· Lech Mazur suite data9%
64Kimi K2· Lech Mazur suite data8.9%
65Claude Sonnet 3.7· Lech Mazur suite data8.7%
66Gemini 1.5 Pro· Lech Mazur suite data8.7%
67Gemini 2.0 Flash· Lech Mazur suite data8.3%
68Claude 3 Opus· Lech Mazur suite data8.2%
69Claude 3.5 Sonnet· Lech Mazur suite data8.1%
70Llama 4 Scout· Lech Mazur suite data7.6%
71Qwen2.5-Max· Lech Mazur suite data7.6%
72DeepSeek V3· Lech Mazur suite data7.4%
73Llama 3.1 405B· Lech Mazur suite data6.7%
74ERNIE 4.5 300B A47B· Lech Mazur suite data6.5%
75DeepSeek V4 Flash· Lech Mazur suite data6.2%
76GPT-4.1 mini· Lech Mazur suite data6.1%
77Llama 3.3 70B Instruct· Lech Mazur suite data5.9%
78MiniMax Text 01· Lech Mazur suite data5.9%
79Mistral Medium 3.1· Lech Mazur suite data5.5%
80Mistral Large 3· Lech Mazur suite data4.8%
81gemma2-27b· Lech Mazur suite data4.3%
82Mistral Small 3.2· Lech Mazur suite data4.3%
83Gemma 3 27B IT· Lech Mazur suite data4.2%
84Mistral Small 3.1· Lech Mazur suite data4.2%
85Nova Pro· Lech Mazur suite data4.1%
86Qwen2.5 72B· Lech Mazur suite data4%
87Claude Haiku 3.5· Lech Mazur suite data3.8%
88GPT-4o mini· Lech Mazur suite data3.4%
89Phi 4· Lech Mazur suite data3.4%
90Mistral-Small-3· Lech Mazur suite data3%
91GPT-4.1 nano· Lech Mazur suite data2.8%
92GLM 4 32B· Lech Mazur suite data2.7%
93Claude 3 Haiku· Lech Mazur suite data0.7%

Showing 51–93 of 93 results.