Skip to content

Benchmark

SkillsBench · Vals · Without Skills · Mean cost / test

Lower scores are better.

Models and systems measured
34
Scores recorded
35
Sources
1
IQ weight
No weight published

Score distribution

1.3
Mean
0.745
Median
1.4
Std dev
0.092
10th percentile
3.9
90th percentile
0.0399
Min
0.03995.3
The band covers the middle 80% of scores. The line marks the median.

Results

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
Claude Fable 5.1Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=anthropic/claude-fable-5-1 · provider=Anthropic · compute_effort=max · temperature=1 · max_output_tokens=128000· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.4.5
Claude Opus 4.8Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=anthropic/claude-opus-4-8 · provider=Anthropic · compute_effort=max · temperature=1 · max_output_tokens=128000· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.5.3
Claude Opus 5Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=anthropic/claude-opus-5 · provider=Anthropic · compute_effort=max · temperature=1 · max_output_tokens=128000· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.2.8
Claude Sonnet 4.6Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=anthropic/claude-sonnet-4-6 · provider=Anthropic · compute_effort=max · reasoning=true · temperature=1 · max_output_tokens=128000· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.1.3
Claude Sonnet 5Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=anthropic/claude-sonnet-5 · provider=Anthropic · compute_effort=max · temperature=1 · max_output_tokens=128000· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.3.9
DeepSeek V4 Flash 0731Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=deepseek/deepseek-v4-flash-0731 · provider=DeepSeek · reasoning_effort=high · max_output_tokens=384000· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.0.152
DeepSeek V4 Pro (version unspecified)Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=deepseek/deepseek-v4-pro · provider=DeepSeek · reasoning_effort=max · max_output_tokens=384000· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.0.417
DeepSeek V4 Pro 0813Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=deepseek/deepseek-v4-pro-0813 · provider=DeepSeek · reasoning_effort=max · max_output_tokens=384000· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.0.424
DeepSeek V4.1 FlashVals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=deepseek/deepseek-v4.1-flash · provider=DeepSeek · reasoning_effort=high · max_output_tokens=384000· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.0.0923
Gemini 3.5 FlashVals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=google/gemini-3.5-flash · provider=Google · reasoning_effort=high · temperature=1 · max_output_tokens=65536· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.1.4
Gemini 3.7 FlashVals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=google/gemini-3.7-flash · provider=Google · reasoning_effort=high · temperature=1 · max_output_tokens=65536· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.1.9
Gemini 3.8 FlashVals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=google/gemini-3.8-flash · provider=Google · reasoning_effort=high · temperature=1 · max_output_tokens=65536· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.2.6
GLM 5.2Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=zai/glm-5.2 · provider=Zhipu AI · reasoning_effort=max · temperature=1 · top_p=0.95 · max_output_tokens=131072· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.0.86
GLM 5.3Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=zai/glm-5.3 · provider=Zhipu AI · reasoning_effort=max · temperature=1 · top_p=0.95 · max_output_tokens=128000· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.0.745
GLM 5.3 FlashVals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=zai/glm-5.3-flash · provider=Fireworks AI · reasoning_effort=max · temperature=1 · top_p=0.95 · max_output_tokens=131072· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.0.0399
GPT 5.4Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=openai/gpt-5.4-2026-03-05 · provider=OpenAI · reasoning_effort=xhigh · max_output_tokens=128000· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.2.1
GPT 5.5Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · Codex (version not stated) · Without Skills · system=openai/gpt-5.5-codex · provider=OpenAI· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.1.5
GPT 5.5Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=openai/gpt-5.5 · provider=OpenAI · reasoning_effort=xhigh · max_output_tokens=128000· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.2.5
GPT-5.6 LunaVals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=openai/gpt-5.6-luna · provider=OpenAI · reasoning_effort=max · max_output_tokens=128000· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.0.251
GPT-5.6 SolVals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=openai/gpt-5.6-sol · provider=OpenAI · reasoning_effort=max · max_output_tokens=128000· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.4.1
GPT-5.6 TerraVals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=openai/gpt-5.6-terra · provider=OpenAI · reasoning_effort=max · max_output_tokens=128000· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.1.9
Grok 4.3Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=grok/grok-4.3 · provider=SpaceXAI · reasoning_effort=high · temperature=0.7 · top_p=0.95· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.0.362
Grok 4.5Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=grok/grok-4.5 · provider=SpaceXAI · reasoning_effort=high · temperature=0.7 · top_p=0.95· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.0.625
Grok 4.6Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=grok/grok-4.6 · provider=SpaceXAI · reasoning_effort=high · temperature=0.7 · top_p=0.95· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.1
InklingVals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=thinkingmachines/inkling · provider=Thinkingmachines · reasoning_effort=0.99 · temperature=1 · top_p=1 · max_output_tokens=256000· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.0.901
Inkling-SmallVals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=thinkingmachines/inkling-small · provider=Thinkingmachines · reasoning_effort=0.99 · temperature=1 · top_p=1 · max_output_tokens=262000· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.0.146
Kimi K2.7 CodeVals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=kimi/kimi-k2.7-code · provider=Moonshot AI · top_p=0.95 · max_output_tokens=32768· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.0.437
Ling-3.0-FlashVals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=ant/ling-3.0-flash-2607 · provider=Ant · temperature=0.6 · top_p=0.95 · max_output_tokens=131072· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.0.0642
Mercury 2.5Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=inception/mercury-2.5 · provider=Inception · reasoning_effort=high · max_output_tokens=65536· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.0.0742
MiniMax M3Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=minimax/MiniMax-M3 · provider=MiniMax · temperature=1 · top_p=0.95 · max_output_tokens=512000· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.0.597
Muse Spark 1.1Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=meta/muse_spark_1_1 · provider=Meta · reasoning_effort=xhigh · max_output_tokens=256000· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.0.462
Muse Spark 1.2Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=meta/muse_spark_1_2 · provider=Meta · reasoning_effort=xhigh · temperature=1 · max_output_tokens=131072· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.0.696
Qwen 3.8 27BVals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=alibaba/qwen3.8-27b · provider=Alibaba · reasoning_effort=xhigh · temperature=1 · top_p=0.95 · max_output_tokens=65536· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.0.829
Qwen3.7 PlusVals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=alibaba/qwen3.7-plus · provider=Alibaba · temperature=0.7 · max_output_tokens=65536· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.0.147
Qwen3.8 MaxVals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=alibaba/qwen3.8-max · provider=Alibaba · temperature=0.7 · max_output_tokens=128000· Reported by Vals AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.0.559

Data credits: BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links.