TestsBenchmarks · Coding
SWE Marathon
Very long-horizon software engineering tasks (resolution rate, pass@1).
Measured by independent testersIndependent Reported by the makerVendor-reported
| Thinking levelSetting | |||||||
|---|---|---|---|---|---|---|---|
| 1 | GLM-5.3 | Z.ai (Zhipu) | 42.5% | Maxreasoning_effort=max | Maker's own figureVendor-reportedZ.ai ↗ | Claude Code 2.1.207 | 14 Aug 2026 |
| 2 | Kimi K3 | Moonshot AI (Kimi) | 42.0% | Maxreasoning_effort=max | Maker's own figureVendor-reportedMoonshot AI ↗ | Claude Code | 16 Jul 2026 |
| 3 | Grok 4.5 | SpaceXAI (formerly xAI) | 29.0% | Defaulteffort not stated (API default high) | Maker's own figureVendor-reportedSpaceXAI ↗ | — | 16 Jul 2026 |
| 4 | GLM-5.2 | Z.ai (Zhipu) | 13.0% | Maxmax effort | Maker's own figureVendor-reportedZ.ai ↗ | — | 16 Jun 2026 |