TestsBenchmarks · Tool use
AutomationBench
Zapier's benchmark of end-to-end business workflows across many connected apps (47 tools).
Measured by independent testersIndependent Reported by the makerVendor-reported
| Thinking levelSetting | |||||||
|---|---|---|---|---|---|---|---|
| 1 | DeepSeek V4.1 Flash | DeepSeek | 54.8% | Maxreasoning_effort=100 (max) | Maker's own figureVendor-reportedDeepSeek ↗ | official scaffold | 10 Sep 2026 |
| 2 | Gemini 4 Argon | Google (Gemini / DeepMind) | 51.3% | Maxhighest thinking settings | Maker's own figureVendor-reportedGoogle ↗ | — | 30 Sep 2026 |
| 3 | Muse Spark 1.3 | Meta (Meta Superintelligence Labs, Muse) | 49.6% | Maxmax reasoning | Maker's own figureVendor-reportedMeta ↗ | — | 2 Sep 2026 |
| 4 | GLM-5.3-Flash | Z.ai (Zhipu) | 48.8% | Maxreasoning_effort=max (default) | Maker's own figureVendor-reportedZ.ai ↗ | — | 26 Aug 2026 |
| 5 | GLM-5.3 | Z.ai (Zhipu) | 48.2% | Maxreasoning_effort=max | Maker's own figureVendor-reportedZ.ai ↗ | — | 14 Aug 2026 |
| 6 | Claude Sonnet 5.5 | Anthropic | 44.7% | Maxadaptive thinking, effort=max | Maker's own figureVendor-reportedAnthropic ↗ | — | 28 Sep 2026 |
| 7 | GPT-6 Astra | OpenAI | 41.4% | Maxreasoning effort=max | Maker's own figureVendor-reportedOpenAI ↗ | — | 29 Sep 2026 |
| 8 | Claude Opus 5.5 | Anthropic | 40.0% | Maxadaptive thinking, effort=max | Maker's own figureVendor-reportedAnthropic ↗ | — | 22 Sep 2026 |
| 9 | Muse Spark 1.2 | Meta (Meta Superintelligence Labs, Muse) | 38.2% | Extra high | Maker's own figureVendor-reportedMeta ↗ | — | 2 Sep 2026 |
| 10 | GPT-6.1 Sol | OpenAI | 36.1% | Maxreasoning effort=max | Maker's own figureVendor-reportedOpenAI ↗ | — | 29 Sep 2026 |
| 11 | GPT-6 Sol | OpenAI | 33.2% | Extra highreasoning effort=xhigh | Maker's own figureVendor-reportedOpenAI ↗ | — | 29 Sep 2026 |
| 12 | DeepSeek V4 Pro (0813) | DeepSeek | 31.8% | Maxreasoning_effort=max | Maker's own figureVendor-reportedDeepSeek ↗ | — | 13 Aug 2026 |
| 13 | Claude Fable 5.1 | Anthropic | 31.4% | Maxadaptive thinking, effort=max | Maker's own figureVendor-reportedAnthropic ↗ | — | 22 Sep 2026 |
| 14 | Kimi K3 | Moonshot AI (Kimi) | 30.8% | Maxreasoning_effort=max | Maker's own figureVendor-reportedMoonshot AI ↗ | — | 16 Jul 2026 |
| 15 | Gemini 3.7 Flash | Google (Gemini / DeepMind) | 30.4% | Defaultdefault settings (API default thinking_level=medium) | Maker's own figureVendor-reportedGoogle ↗ | — | 13 Aug 2026 |
| 16 | GPT-5.6 Sol | OpenAI | 28.8% | Maxreasoning effort=max | Maker's own figureVendor-reportedOpenAI ↗ | — | 22 Sep 2026 |
| 17 | Qwen3.8 Max (0803) | Qwen (Alibaba) | 27.3% | Defaultthinking (default reasoning_effort=xhigh; eval setting not stated) | Maker's own figureVendor-reportedQwen (Alibaba) ↗ | — | 3 Aug 2026 |
| 18 | Claude Opus 5 | Anthropic | 26.9% | Maxadaptive thinking, effort=max | Maker's own figureVendor-reportedAnthropic ↗ | — | 22 Sep 2026 |
| 19 | GLM-5.2 | Z.ai (Zhipu) | 26.2% | Maxsetting not stated (GLM-5.3 blog comparison column) | Maker's own figureVendor-reportedZ.ai ↗ | — | 14 Aug 2026 |
| 20 | DeepSeek V4 Flash Vision Exp | DeepSeek | 25.7% | Maxreasoning_effort=max | Maker's own figureVendor-reportedDeepSeek ↗ | — | 21 Aug 2026 |
| 21 | DeepSeek V4 Flash (0731) | DeepSeek | 25.1% | Maxreasoning_effort=max | Maker's own figureVendor-reportedDeepSeek ↗ | — | 31 Jul 2026 |
| 22 | GPT-6 Luna | OpenAI | 20.7% | Maxreasoning effort=max | Maker's own figureVendor-reportedOpenAI ↗ | — | 22 Sep 2026 |
| 23 | Claude Fable 5 | Anthropic | 17.1% | Maxadaptive thinking, effort=max | Maker's own figureVendor-reportedAnthropic ↗ | — | 1 Sep 2026 |
| 24 | Claude Opus 4.8 | Anthropic | 17.0% | Maxadaptive thinking, effort=max | Maker's own figureVendor-reportedAnthropic ↗ | — | 24 Jul 2026 |
| 25 | GPT-5.6 Luna | OpenAI | 17.0% | Maxreasoning effort=max | Maker's own figureVendor-reportedOpenAI ↗ | — | 22 Sep 2026 |
| 26 | Gemini 3.6 Flash | Google (Gemini / DeepMind) | 17.0% | Defaultdefault settings (API default thinking_level=medium) | Maker's own figureVendor-reportedGoogle ↗ | — | 13 Aug 2026 |
| 27 | GPT-5.6 Terra | OpenAI | 15.2% | DefaultGPT-5.6 launch table (effort not stated; text cites max for headline results) | Maker's own figureVendor-reportedOpenAI ↗ | — | 9 Jul 2026 |
| 28 | Qwen3.7 Max | Qwen (Alibaba) | 14.2% | Defaultthinking (setting not stated) | Maker's own figureVendor-reportedQwen (Alibaba) ↗ | — | 3 Aug 2026 |
| 29 | GPT-5.5 | OpenAI | 12.9% | DefaultGPT-5.6 launch table (effort not stated; text cites max for headline results) | Maker's own figureVendor-reportedOpenAI ↗ | — | 9 Jul 2026 |
| 30 | DeepSeek V4 Pro (Preview, 0423) | DeepSeek | 12.8% | Defaultsetting not stated for preview columns | Maker's own figureVendor-reportedDeepSeek ↗ | — | 13 Aug 2026 |
| 31 | DeepSeek V4 Flash (Preview, 0423) | DeepSeek | 10.8% | Defaultsetting not stated for preview columns | Maker's own figureVendor-reportedDeepSeek ↗ | — | 13 Aug 2026 |
| 32 | Claude Sonnet 5 | Anthropic | 10.7% | Maxadaptive thinking, effort=max | Maker's own figureVendor-reportedAnthropic ↗ | — | 28 Sep 2026 |
| 33 | Claude Sonnet 4.6 | Anthropic | 5.3% | Maxadaptive thinking, effort=max | Maker's own figureVendor-reportedAnthropic ↗ | — | 30 Jun 2026 |