TestsBenchmarks · Coding
ProgramBench (Almost Solved)
ProgramBench reported as the share of tasks 'almost solved' (Almost@1); a much stricter metric than the default ProgramBench score.
Measured by independent testersIndependent Reported by the makerVendor-reported
| Thinking levelSetting | |||||||
|---|---|---|---|---|---|---|---|
| 1 | DeepSeek V4.1 Flash | DeepSeek | 20.3% | Maxreasoning_effort=100 (max) | Maker's own figureVendor-reportedDeepSeek ↗ | DeepSeek Harness (Minimal) | 10 Sep 2026 |
| 2 | GLM-5.3 | Z.ai (Zhipu) | 19.0% | Maxreasoning_effort=max | Maker's own figureVendor-reportedZ.ai ↗ | — | 14 Aug 2026 |
| 3 | DeepSeek V4 Pro (0813) | DeepSeek | 15.5% | MaxMax reasoning effort | Maker's own figureVendor-reportedDeepSeek ↗ | DeepSeek Harness (minimal mode) | 10 Sep 2026 |
| 4 | GLM-5.2 | Z.ai (Zhipu) | 9.5% | Maxsetting not stated (GLM-5.3 blog comparison column) | Maker's own figureVendor-reportedZ.ai ↗ | — | 14 Aug 2026 |