Why Language Models Hallucinate

Hallucinations stem from training incentives, not inherent model flaws: Language models guess when uncertain because their training rewards confident answers over honest admissions of uncertainty—similar to how students might guess on difficult exam questions rather than leaving answers blank.

The root cause is statistical: models cannot distinguish fact from fiction during training: When language models are trained on text data, they learn to predict plausible-sounding words without necessarily learning to distinguish correct statements from incorrect ones, making hallucinations a natural byproduct of the training process.

Evaluation methods perpetuate the problem: Current benchmark scoring systems penalize models for expressing uncertainty, incentivizing them to guess rather than say "I don't know." This creates what the authors call an "epidemic of penalizing uncertain responses" that pushes models toward unreliable behavior.

The solution requires changing how we measure success: Rather than creating new hallucination tests, the field should modify existing benchmark scoring systems to reward honest uncertainty. This socio-technical fix—changing what we measure on leaderboards—could steer AI development toward more trustworthy systems.

This problem is fixable through better alignment of incentives: Hallucinations are not mysterious or inevitable; they result from misaligned goals between what we want (reliable AI) and what we reward (high test scores), suggesting the issue can be addressed through deliberate changes to evaluation practices.