TestsBenchmarks · Knowledge
Vectara Hallucination Leaderboard (HHEM)
Share of document summaries that contain claims not supported by the source document, judged by Vectara HHEM-2.3.
Measured by independent testersIndependent Reported by the makerVendor-reported
Lower is better here: the shortest bar is the best result.
| Thinking levelSetting | |||||||
|---|---|---|---|---|---|---|---|
| 1 | Gemma 4 26B A4B IT | Google (Gemini / DeepMind) | 5.2% | Defaultgoogle/gemma-4-26b-a4b-it | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 2 | GPT-5.4 Mini | OpenAI | 5.5% | Defaultopenai/gpt-5.4-mini-2026-03-17 | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 3 | DeepSeek V3.2 | DeepSeek | 6.3% | Defaultdeepseek-ai/DeepSeek-V3.2 | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 4 | GPT-6 Sol | OpenAI | 6.5% | Defaultopenai/gpt-6-sol | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 5 | GPT-5.4 | OpenAI | 7.0% | Defaultopenai/gpt-5.4-2026-03-05 | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 6 | Gemma 4 31B IT | Google (Gemini / DeepMind) | 7.4% | Defaultgoogle/gemma-4-31b-it | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 7 | Gemini 3.1 Flash-Lite | Google (Gemini / DeepMind) | 8.2% | Defaultgoogle/gemini-3.1-flash-lite-preview | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 8 | GPT-5.4 Pro | OpenAI | 8.3% | Defaultopenai/gpt-5.4-pro-2026-03-05 | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 9 | DeepSeek V4 Pro (Preview, 0423) | DeepSeek | 8.6% | Defaultdeepseek-ai/DeepSeek-V4-Pro | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 10 | GPT-6 Astra | OpenAI | 8.7% | Defaultopenai/gpt-6-astra | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 11 | MiniMax M2.5 | MiniMax | 9.1% | DefaultMiniMaxAI/minimax-m2p5 | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 12 | GPT-5.5 | OpenAI | 9.3% | Defaultopenai/gpt-5.5 | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 13 | Claude Haiku 4.5 | Anthropic | 9.8% | Defaultanthropic/claude-haiku-4-5-20251001 | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 14 | GLM 5 | Z.ai (Zhipu) | 10.1% | Defaultzai-org/glm-5 | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 15 | Gemini 3.1 Pro (Preview) | Google (Gemini / DeepMind) | 10.4% | Defaultgoogle/gemini-3.1-pro-preview | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 16 | Qwen3.5-35B-A3B | Qwen (Alibaba) | 10.5% | Defaultqwen/qwen3.5-35b-a3b | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 17 | Claude Sonnet 4.6 | Anthropic | 10.6% | Defaultanthropic/claude-sonnet-4-6 | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 18 | Kimi K2.6 | Moonshot AI (Kimi) | 10.8% | Defaultmoonshotai/kimi-k2.6 | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 19 | Claude Opus 4.5 | Anthropic | 10.9% | Defaultanthropic/claude-opus-4-5-20251101 | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 20 | MiniMax M2.1 | MiniMax | 11.8% | DefaultMiniMaxAI/minimax-m2p1 | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 21 | Claude Sonnet 4.5 | Anthropic | 12.0% | Defaultanthropic/claude-sonnet-4-5-20250929 | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 22 | Claude Opus 4.7 | Anthropic | 12.0% | Defaultanthropic/claude-opus-4-7 | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 23 | Claude Opus 4.6 | Anthropic | 12.2% | Defaultanthropic/claude-opus-4-6 | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 24 | GPT-5.6 Sol | OpenAI | 12.4% | Defaultopenai/gpt-5.6-sol | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 25 | MiniMax M2.7 | MiniMax | 12.9% | DefaultMiniMaxAI/minimax-m2p7 | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 26 | Gemini 3 Flash Preview | Google (Gemini / DeepMind) | 13.5% | Defaultgoogle/gemini-3-flash-preview | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 27 | Gemini 3 Pro | Google (Gemini / DeepMind) | 13.6% | Defaultgoogle/gemini-3-pro-preview | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |
| 28 | Kimi K2.5 | Moonshot AI (Kimi) | 14.2% | Defaultmoonshotai/Kimi-K2.5 | Independent testIndependentVectara ↗ | — | 22 Sep 2026 |