Benchmark
CodeMemoryBench · Code resolved
Coding benchmarks compare models on programming tasks. Scores are percentages. Higher is better.
- Models and systems measured
- 2
- Scores recorded
- 14
- Sources
- 1
- IQ weight
- No weight published
Score distribution
- 30%
- Mean
- 29.2%
- Median
- 11.9%
- Std dev
- 13%
- 10th percentile
- 43.1%
- 90th percentile
- 11.6%
- Min
Results
The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.
| Model or system | Score |
|---|---|
| Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: FullContext · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs | 58.8% |
| Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: Mem0 · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs | 36.6% |
| Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: MemOS · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs | 34.7% |
| Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: OpenViking · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs | 29.2% |
| Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: RAG · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs | 34.7% |
| Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: RepoMemory · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs | 17.6% |
| Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: TriLink · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs | 43.1% |
| DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: FullContext · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs | 28.2% |
| DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: Mem0 · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs | 21.8% |
| DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: MemOS · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs | 28.2% |
| DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: OpenViking · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs | 13% |
| DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: RAG · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs | 26.9% |
| DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: RepoMemory · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs | 11.6% |
| DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: TriLink · candidate snapshot, reasoning effort, and run count not stated · 216 Code tasks; F2P and P2P required· Reported by Tencent Hunyuan· via BenchBoard docs | 35.6% |