Skip to content

Benchmark

OSWorld

Agentic benchmarks measure how well models carry out tasks that take several steps. Scores are percentages.

Models measured
29
Scores recorded
31
Sources
3
IQ weight
1.34

Score distribution

54.5%
Mean
62.3%
Median
23.3%
Std dev
23%
10th percentile
81.2%
90th percentile
5%
Min
5%86.1%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.34 for chat models, measured over 29 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Qwen3.8 Max· Steel.dev Agent Leaderboards data86.1%
2Claude Fable 5· Steel.dev Agent Leaderboards data85%
3Claude Opus 4.8· Steel.dev Agent Leaderboards data83.4%
4MiMo V2.6 Pro· Steel.dev Agent Leaderboards data82%
5Claude Sonnet 5· Steel.dev Agent Leaderboards data81.2%
6MiMo V2.6 Flash· Steel.dev Agent Leaderboards data80.8%
7Doubao Seed 2.1 Pro· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim78.8%
8Claude Sonnet 4.6· Steel.dev Agent Leaderboards data
Values from 3 sources
78.5%
9Doubao Seed 2.1 Turbo· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim76.4%
10GPT 5.4· Steel.dev Agent Leaderboards data75%
11Claude Opus 4.6· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim
Values from 2 sources
72.7%
12Qwen3 VL 235B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.
Values from 2 sources
66.7%
13Claude Opus 4.5· Epoch AI Benchmarking Hub data
Values from 3 sources
66.3%
14Kimi K2.5· Epoch AI Benchmarking Hub data
Values from 2 sources
63.3%
15Claude Sonnet 4.5· Epoch AI Benchmarking Hub data
Values from 3 sources
62.9%
16GLM 5V Turbo· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim
Values from 2 sources
62.3%
17Doubao Seed 1.8· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim61.9%
18Claude Haiku 4.5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim
Values from 2 sources
50.7%
19Claude Sonnet 4· Epoch AI Benchmarking Hub data43.9%
20Qwen3 VL 32B Thinking· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.41%
21Qwen3 VL 235Bthinking· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.38.1%
22Claude Sonnet 3.7· Epoch AI Benchmarking Hub data35.8%
23Qwen: Qwen3 VL 8B Thinking· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.33.9%
24Qwen3 VL 8B Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.33.9%
25Qwen3 VL 32B Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.32.6%
26Computer Use Preview· Epoch AI Benchmarking Hub data31.3%
27qwen3-vl-30bthinking· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.30.6%
28qwen3-vl-30b· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.30.3%
29OpenAI o3medium· Epoch AI Benchmarking Hub data23%
30Qwen2.5 VL 72B Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim
Values from 3 sources
8.8%
31Qwen 2.5 VL 32B Instruct· Steel.dev Agent Leaderboards data5.9%

Data credits: Steel.dev Agent Leaderboards, MIT (licence) · Data by LLM Stats · Epoch AI Benchmarking Hub, CC BY 4.0 (licence)