TestsBenchmarks · Tool use
Toolathlon
Multi-app tool-use tasks; Pass@1.
Measured by independent testersIndependent Reported by the makerVendor-reported
| Thinking levelSetting | |||||||
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5 | Anthropic | 80.6% | Maxadaptive thinking, effort=max | Maker's own figureVendor-reportedAnthropic ↗ | — | 22 Sep 2026 |
| 2 | Claude Opus 4.8 | Anthropic | 79.9% | Maxadaptive thinking, effort=max | Maker's own figureVendor-reportedAnthropic ↗ | — | 22 Sep 2026 |
| 3 | Claude Opus 5.5 | Anthropic | 77.8% | Maxadaptive thinking, effort=max | Maker's own figureVendor-reportedAnthropic ↗ | — | 22 Sep 2026 |
| 4 | Claude Fable 5.1 | Anthropic | 77.8% | Maxadaptive thinking, effort=max | Maker's own figureVendor-reportedAnthropic ↗ | — | 22 Sep 2026 |
| 5 | Claude Sonnet 5 | Anthropic | 74.7% | Maxadaptive thinking, effort=max | Maker's own figureVendor-reportedAnthropic ↗ | — | 22 Sep 2026 |
| 6 | GPT-5.6 Sol | OpenAI | 58.0% | DefaultGPT-5.6 launch table (effort not stated; text cites max for headline results) | Maker's own figureVendor-reportedOpenAI ↗ | — | 9 Jul 2026 |
| 7 | Gemini 3.5 Flash | Google (Gemini / DeepMind) | 56.5% | Defaultdefault settings (API default thinking_level=medium) | Maker's own figureVendor-reportedGoogle ↗ | — | 19 May 2026 |
| 8 | GPT-5.5 | OpenAI | 55.6% | DefaultGPT-5.6 launch table (effort not stated; text cites max for headline results) | Maker's own figureVendor-reportedOpenAI ↗ | — | 9 Jul 2026 |
| 9 | GPT-5.4 | OpenAI | 54.6% | Extra highreasoning effort=xhigh | Maker's own figureVendor-reportedOpenAI ↗ | — | 23 Apr 2026 |
| 10 | GPT-5.6 Luna | OpenAI | 53.4% | DefaultGPT-5.6 launch table (effort not stated; text cites max for headline results) | Maker's own figureVendor-reportedOpenAI ↗ | — | 9 Jul 2026 |
| 11 | GPT-5.6 Terra | OpenAI | 53.1% | DefaultGPT-5.6 launch table (effort not stated; text cites max for headline results) | Maker's own figureVendor-reportedOpenAI ↗ | — | 9 Jul 2026 |
| 12 | DeepSeek V4 Pro (Preview, 0423) | DeepSeek | 51.8% | MaxThink Max | Maker's own figureVendor-reportedDeepSeek ↗ | — | 24 Apr 2026 |
| 13 | Kimi K2.6 | Moonshot AI (Kimi) | 50.0% | Defaultthinking | Maker's own figureVendor-reportedMoonshot AI ↗ | — | 20 Apr 2026 |
| 14 | GLM-5.2 | Z.ai (Zhipu) | 48.2% | Defaultthinking (effort not stated) | Maker's own figureVendor-reportedZ.ai ↗ | — | 16 Jun 2026 |
| 15 | DeepSeek V4 Flash (Preview, 0423) | DeepSeek | 47.8% | MaxThink Max | Maker's own figureVendor-reportedDeepSeek ↗ | — | 24 Apr 2026 |
| 16 | MiniMax M2.7 | MiniMax | 46.3% | Defaultsetting not stated | Maker's own figureVendor-reportedMiniMax ↗ | — | 18 Mar 2026 |
| 17 | GLM-5.1 | Z.ai (Zhipu) | 40.7% | Defaultthinking | Maker's own figureVendor-reportedZ.ai ↗ | — | 7 Apr 2026 |
| 18 | Qwen3.6 Plus | Qwen (Alibaba) | 39.8% | Defaultthinking | Maker's own figureVendor-reportedQwen (Alibaba) ↗ | — | 2 Apr 2026 |