Poly-FEVER: A Multilingual Fact Verification Benchmark for Hallucination Detection in Large Language Models

Hallucinations are a widespread multilingual problem: The Poly-FEVER benchmark reveals that hallucinations—where LLMs generate false information—occur across all 11 languages tested, not just in English, indicating this is a systemic challenge for global AI deployment rather than an isolated issue.

Language and resource availability create unequal AI reliability: The research found that "topic distribution and web resource availability influence hallucination frequency," meaning LLMs perform more reliably in languages with more internet content, creating disparities where speakers of less-resourced languages receive less trustworthy AI systems.

Popular models like ChatGPT and LLaMA show language-specific biases: The study systematically evaluated major commercial and open-source LLMs across languages, uncovering that these models have inconsistent accuracy rates depending on the language, suggesting their training data introduces systematic blind spots.

Existing evaluation tools are inadequate for global AI assessment: Prior to Poly-FEVER, fact-checking benchmarks "focus primarily on English and a few widely spoken languages," meaning developers lacked the tools to properly test whether their AI systems work reliably for speakers of less common languages—a critical gap for responsible AI development.

Transparent multilingual benchmarks enable accountability: By making the 77,973-claim dataset publicly available, Poly-FEVER allows independent researchers and developers to identify and address hallucination problems across languages, supporting the development of more equitable and trustworthy AI systems.