TestsBenchmarks · Coding
SWE-bench Verified (bash-only, mini-SWE-agent)
SWE-bench Verified run by the SWE-bench team with the minimal bash-only mini-SWE-agent so models are compared on equal scaffolding.
% solvedHigher is betterToo easy now: the top models all score near perfectSaturatedThe test's websiteOfficial page ↗
Measured by independent testersIndependent Reported by the makerVendor-reported
| Thinking levelSetting | |||||||
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 4.5 | Anthropic | 76.8% | High | Independent testIndependentSWE-bench ↗ | mini-SWE-agent 2.0.0 | 17 Feb 2026 |
| 2 | Gemini 3 Flash Preview | Google (Gemini / DeepMind) | 75.8% | High | Independent testIndependentSWE-bench ↗ | mini-SWE-agent 2.0.0 | 17 Feb 2026 |
| 3 | MiniMax M2.5 | MiniMax | 75.8% | High | Independent testIndependentSWE-bench ↗ | mini-SWE-agent 2.0.0 | 17 Feb 2026 |
| 4 | Claude Opus 4.6 | Anthropic | 75.6% | Default | Independent testIndependentSWE-bench ↗ | mini-SWE-agent 2.0.0 | 17 Feb 2026 |
| 5 | GLM 5 | Z.ai (Zhipu) | 72.8% | High | Independent testIndependentSWE-bench ↗ | mini-SWE-agent 2.0.0 | 17 Feb 2026 |
| 6 | GPT-5.2 | OpenAI | 72.8% | High | Independent testIndependentSWE-bench ↗ | mini-SWE-agent 2.0.0 | 17 Feb 2026 |
| 7 | GPT-5.2-Codex | OpenAI | 72.8% | Default | Independent testIndependentSWE-bench ↗ | mini-SWE-agent 2.0.0 | 19 Feb 2026 |
| 8 | Claude Sonnet 4.5 | Anthropic | 71.4% | High | Independent testIndependentSWE-bench ↗ | mini-SWE-agent 2.0.0 | 17 Feb 2026 |
| 9 | Kimi K2.5 | Moonshot AI (Kimi) | 70.8% | High | Independent testIndependentSWE-bench ↗ | mini-SWE-agent 2.0.0 | 17 Feb 2026 |
| 10 | DeepSeek V3.2 | DeepSeek | 70.0% | High | Independent testIndependentSWE-bench ↗ | mini-SWE-agent 2.0.0 | 17 Feb 2026 |
| 11 | Gemini 3 Pro | Google (Gemini / DeepMind) | 69.6% | High | Independent testIndependentSWE-bench ↗ | mini-SWE-agent 2.0.0 | 26 Feb 2026 |
| 12 | Claude Haiku 4.5 | Anthropic | 66.6% | High | Independent testIndependentSWE-bench ↗ | mini-SWE-agent 2.0.0 | 17 Feb 2026 |
| 13 | GPT-5 Mini | OpenAI | 56.2% | Default | Independent testIndependentSWE-bench ↗ | mini-SWE-agent 2.0.0 | 17 Feb 2026 |