TestsBenchmarks · Multimodal

CharXiv Reasoning (with tools)

CharXiv Reasoning with search/code tools.

% solvedHigher is betterThe test's websiteOfficial page ↗

The best 5 · 5 models tested

Top 5 · best setting per model · 5 models, 5 results (0 independent)

Measured by independent testersIndependent Reported by the makerVendor-reported

CharXiv Reasoning (with tools) leaderboard
				Thinking levelSetting
1	Gemini 3.6 Flash	Google (Gemini / DeepMind)	89.4%	Defaultdefault settings (API default thinking_level=medium)	Maker's own figureVendor-reportedGoogle ↗	—	21 Jul 2026
2	Gemini 3.7 Flash	Google (Gemini / DeepMind)	88.7%	Defaultdefault settings (API default thinking_level=medium)	Maker's own figureVendor-reportedGoogle ↗	—	13 Aug 2026
3	Gemini 3.5 Flash	Google (Gemini / DeepMind)	84.9%	Defaultdefault settings (API default thinking_level=medium)	Maker's own figureVendor-reportedGoogle ↗	—	21 Jul 2026
4	Gemini 3.5 Flash-Lite	Google (Gemini / DeepMind)	76.5%	Highhigh thinking	Maker's own figureVendor-reportedGoogle ↗	—	21 Jul 2026
5	Gemini 3.1 Flash-Lite	Google (Gemini / DeepMind)	75.6%	Highhigh thinking	Maker's own figureVendor-reportedGoogle ↗	—	21 Jul 2026