Skip to content

Benchmark

LiveMCPBench

Agentic benchmarks measure how well models carry out tasks that take several steps. Scores are percentages. Higher is better.

Models measured
13
Scores recorded
13
Sources
1
IQ weight
1.92

Score distribution

48.1%
Mean
44.2%
Median
13.6%
Std dev
32.6%
10th percentile
70.5%
90th percentile
30.5%
Min
30.5%79%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.92 for chat models, measured over 13 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Sonnet 4Claude-Sonnet-4-20250514, LiveMCPBench· LiveMCPBench leaderboard data docs79%
2Claude Opus 4Claude-Opus-4-20250514, LiveMCPBench· LiveMCPBench leaderboard data docs70.5%
3GLM 4.5LiveMCPBench· LiveMCPBench leaderboard data docs61%
4DeepSeek R1Deepseek-R1-0528, LiveMCPBench· LiveMCPBench leaderboard data docs48.4%
5Qwen 3 235B A22BQwen3-235B-A22B, LiveMCPBench· LiveMCPBench leaderboard data docs48.4%
6GPT-5 miniGPT-5-Mini, medium, LiveMCPBench· LiveMCPBench leaderboard data docs45.3%
7GPT-4.1 miniGPT-4.1-Mini, LiveMCPBench· LiveMCPBench leaderboard data docs44.2%
8Qwen2.5 72B InstructQwen2.5-72B-Instruct, LiveMCPBench· LiveMCPBench leaderboard data docs43.2%
9DeepSeek V3Deepseek-V3-0324, LiveMCPBench· LiveMCPBench leaderboard data docs42.1%
10Gemini 2.5 ProGemini-2.5-Pro, LiveMCPBench· LiveMCPBench leaderboard data docs41.1%
11GPT 4.1GPT-4.1, LiveMCPBench· LiveMCPBench leaderboard data docs39%
12Kimi K2Kimi-K2, LiveMCPBench· LiveMCPBench leaderboard data docs32.6%
13Qwen3 32BQwen3-32B, LiveMCPBench· LiveMCPBench leaderboard data docs30.5%

Data credits: LiveMCPBench leaderboard, ICIP-CAS (Mo et al., 2025). Sheet linked from the Apache-2.0 project README; sheet licence not separately stated.