TestsBenchmarks · Coding
Vals SRE Bench
Share of contamination-free real-world binaries an agent can fully reverse-engineer.
Measured by independent testersIndependent Reported by the makerVendor-reported
| Thinking levelSetting | |||||||
|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | OpenAI | 56.9% | Maxreasoning_effort=max | Independent testIndependentVals.ai ↗ | — | 29 Sep 2026 |
| 2 | GPT-6.1 Sol | OpenAI | 50.8% | Maxreasoning_effort=max | Independent testIndependentVals.ai ↗ | — | 29 Sep 2026 |
| 3 | Gemini 4 Argon | Google (Gemini / DeepMind) | 44.3% | Highreasoning_effort=high | Independent testIndependentVals.ai ↗ | — | 29 Sep 2026 |
| 4 | Claude Opus 5.5 | Anthropic | 33.6% | Maxeffort=max | Independent testIndependentVals.ai ↗ | — | 29 Sep 2026 |
| 5 | GPT-5.6 Sol | OpenAI | 30.5% | Maxreasoning_effort=max | Independent testIndependentVals.ai ↗ | — | 29 Sep 2026 |
| 6 | Claude Sonnet 5.5 | Anthropic | 30.1% | Maxeffort=max | Independent testIndependentVals.ai ↗ | — | 29 Sep 2026 |
| 7 | Claude Fable 5.1 | Anthropic | 22.9% | Maxeffort=max | Independent testIndependentVals.ai ↗ | — | 29 Sep 2026 |
| 8 | Claude Opus 5 | Anthropic | 12.2% | Maxeffort=max | Independent testIndependentVals.ai ↗ | — | 29 Sep 2026 |
| 9 | Gemini 3.7 Flash | Google (Gemini / DeepMind) | 4.6% | Highreasoning_effort=high | Independent testIndependentVals.ai ↗ | — | 29 Sep 2026 |
| 10 | MiMo-V2.6-Flash | Xiaomi MiMo | 4.2% | Default | Independent testIndependentVals.ai ↗ | — | 29 Sep 2026 |
| 11 | GPT-5.5 | OpenAI | 3.8% | Extra highreasoning_effort=xhigh | Independent testIndependentVals.ai ↗ | — | 29 Sep 2026 |
| 12 | MiMo-V2.6-Pro | Xiaomi MiMo | 3.0% | Default | Independent testIndependentVals.ai ↗ | — | 29 Sep 2026 |
| 13 | Hy4 preview | Tencent Hunyuan | 2.3% | Default | Independent testIndependentVals.ai ↗ | — | 29 Sep 2026 |
| 14 | DeepSeek V4.1 Flash | DeepSeek | 0.8% | Maxreasoning_effort=max | Independent testIndependentVals.ai ↗ | — | 29 Sep 2026 |
| 15 | Grok 4.5 | SpaceXAI (formerly xAI) | 0.8% | Highreasoning_effort=high | Independent testIndependentVals.ai ↗ | — | 29 Sep 2026 |
| 16 | GLM-5.2 | Z.ai (Zhipu) | 0.0% | Maxreasoning_effort=max | Independent testIndependentVals.ai ↗ | — | 29 Sep 2026 |