Benchmark
CodeMemoryBench · QA score
Scores are percentages. Higher is better.
Benchmark sourceScores from BenchBoard docs.
- Models and systems measured
- 2
- Scores recorded
- 14
- Sources
- 1
- IQ weight
- No weight published
Score distribution
- 45.5%
- Mean
- 51.4%
- Median
- 15.1%
- Std dev
- 18.7%
- 10th percentile
- 62.2%
- 90th percentile
- 9.6%
- Min
Results
The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.
| Model or system | Score |
|---|---|
| Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: FullContext · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs | 56.1% |
| Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: Mem0 · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs | 37.8% |
| Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: MemOS · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs | 51.9% |
| Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: OpenViking · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs | 46.8% |
| Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: RAG · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs | 48% |
| Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: RepoMemory · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs | 18.7% |
| Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: TriLink · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs | 58.2% |
| DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: FullContext · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs | 53.4% |
| DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: Mem0 · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs | 40% |
| DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: MemOS · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs | 51.4% |
| DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: OpenViking · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs | 38.6% |
| DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: RAG · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs | 62.2% |
| DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: RepoMemory · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs | 9.6% |
| DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: TriLink · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs | 64.2% |