TestsBenchmarks · Agentic
PaperBench
Reproduce ML research papers end to end, graded with expert rubrics.
Measured by independent testersIndependent Reported by the makerVendor-reported
| Thinking levelSetting | |||||||
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.8 Max (0803) | Qwen (Alibaba) | 93.0% | Defaultthinking (default reasoning_effort=xhigh; eval setting not stated) | Maker's own figureVendor-reportedQwen (Alibaba) ↗ | BasicAgent (Code-Dev) | 3 Aug 2026 |
| 2 | MiniMax M3 | MiniMax | 52.6% | Defaultsetting not stated | Maker's own figureVendor-reportedMiniMax ↗ | Claude Code (Ralph-Loop, 12h) | 1 Jun 2026 |