Skip to content

Benchmark

CodeMemoryBench · Code resolved

Coding benchmarks compare models on programming tasks. Scores are percentages. Higher is better.

Models and systems measured
2
Scores recorded
14
Sources
1
IQ weight
No weight published

Score distribution

30%
Mean
29.2%
Median
11.9%
Std dev
13%
10th percentile
43.1%
90th percentile
11.6%
Min
11.6%58.8%
The band covers the middle 80% of scores. The line marks the median.

Results

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: FullContext · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs58.8%
Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: Mem0 · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs36.6%
Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: MemOS · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs34.7%
Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: OpenViking · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs29.2%
Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: RAG · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs34.7%
Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: RepoMemory · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs17.6%
Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: TriLink · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs43.1%
DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: FullContext · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs28.2%
DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: Mem0 · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs21.8%
DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: MemOS · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs28.2%
DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: OpenViking · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs13%
DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: RAG · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs26.9%
DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: RepoMemory · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs11.6%
DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: TriLink · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs35.6%

Data credits: BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links.