Skip to content

Benchmark

SWE-ContextBench Lite Related · Resolved · CMB paper run

Scores are percentages. Higher is better.

Models and systems measured
2
Scores recorded
14
Sources
1
IQ weight
No weight published

Score distribution

32%
Mean
36.4%
Median
9.8%
Std dev
19.2%
10th percentile
45.5%
90th percentile
19.2%
Min
19.2%47.5%
The band covers the middle 80% of scores. The line marks the median.

Results

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
Claude Sonnet 5 (CMB paper label)SWE-ContextBench Lite Related · CMB paper run · 99 tasks · mini-swe-agent · memory: FullContext · F2P-based resolution · candidate snapshot, reasoning effort, and run count not stated· Reported by Tencent Hunyuan· via BenchBoard docs37.4%
Claude Sonnet 5 (CMB paper label)SWE-ContextBench Lite Related · CMB paper run · 99 tasks · mini-swe-agent · memory: Mem0 · F2P-based resolution · candidate snapshot, reasoning effort, and run count not stated· Reported by Tencent Hunyuan· via BenchBoard docs38.4%
Claude Sonnet 5 (CMB paper label)SWE-ContextBench Lite Related · CMB paper run · 99 tasks · mini-swe-agent · memory: MemOS · F2P-based resolution · candidate snapshot, reasoning effort, and run count not stated· Reported by Tencent Hunyuan· via BenchBoard docs39.4%
Claude Sonnet 5 (CMB paper label)SWE-ContextBench Lite Related · CMB paper run · 99 tasks · mini-swe-agent · memory: NoMemory · F2P-based resolution · candidate snapshot, reasoning effort, and run count not stated· Reported by Tencent Hunyuan· via BenchBoard docs43.4%
Claude Sonnet 5 (CMB paper label)SWE-ContextBench Lite Related · CMB paper run · 99 tasks · mini-swe-agent · memory: OpenViking · F2P-based resolution · candidate snapshot, reasoning effort, and run count not stated· Reported by Tencent Hunyuan· via BenchBoard docs36.4%
Claude Sonnet 5 (CMB paper label)SWE-ContextBench Lite Related · CMB paper run · 99 tasks · mini-swe-agent · memory: RepoMemory · F2P-based resolution · candidate snapshot, reasoning effort, and run count not stated· Reported by Tencent Hunyuan· via BenchBoard docs45.5%
Claude Sonnet 5 (CMB paper label)SWE-ContextBench Lite Related · CMB paper run · 99 tasks · mini-swe-agent · memory: TriLink · F2P-based resolution · candidate snapshot, reasoning effort, and run count not stated· Reported by Tencent Hunyuan· via BenchBoard docs47.5%
DeepSeek V4 Flash (CMB paper label)SWE-ContextBench Lite Related · CMB paper run · 99 tasks · mini-swe-agent · memory: FullContext · F2P-based resolution · candidate snapshot, reasoning effort, and run count not stated· Reported by Tencent Hunyuan· via BenchBoard docs23.2%
DeepSeek V4 Flash (CMB paper label)SWE-ContextBench Lite Related · CMB paper run · 99 tasks · mini-swe-agent · memory: Mem0 · F2P-based resolution · candidate snapshot, reasoning effort, and run count not stated· Reported by Tencent Hunyuan· via BenchBoard docs25.3%
DeepSeek V4 Flash (CMB paper label)SWE-ContextBench Lite Related · CMB paper run · 99 tasks · mini-swe-agent · memory: MemOS · F2P-based resolution · candidate snapshot, reasoning effort, and run count not stated· Reported by Tencent Hunyuan· via BenchBoard docs25.3%
DeepSeek V4 Flash (CMB paper label)SWE-ContextBench Lite Related · CMB paper run · 99 tasks · mini-swe-agent · memory: NoMemory · F2P-based resolution · candidate snapshot, reasoning effort, and run count not stated· Reported by Tencent Hunyuan· via BenchBoard docs20.2%
DeepSeek V4 Flash (CMB paper label)SWE-ContextBench Lite Related · CMB paper run · 99 tasks · mini-swe-agent · memory: OpenViking · F2P-based resolution · candidate snapshot, reasoning effort, and run count not stated· Reported by Tencent Hunyuan· via BenchBoard docs19.2%
DeepSeek V4 Flash (CMB paper label)SWE-ContextBench Lite Related · CMB paper run · 99 tasks · mini-swe-agent · memory: RepoMemory · F2P-based resolution · candidate snapshot, reasoning effort, and run count not stated· Reported by Tencent Hunyuan· via BenchBoard docs19.2%
DeepSeek V4 Flash (CMB paper label)SWE-ContextBench Lite Related · CMB paper run · 99 tasks · mini-swe-agent · memory: TriLink · F2P-based resolution · candidate snapshot, reasoning effort, and run count not stated· Reported by Tencent Hunyuan· via BenchBoard docs28.3%

Data credits: BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links.