Visual Hallucination: Definition, Quantification, and Prescriptive Remediations

Visual hallucination in AI image systems is a major barrier to responsible AI development. The paper identifies hallucination as "perhaps the most significant impediment to the advancement of responsible AI," indicating this is not a minor technical glitch but a fundamental challenge that must be addressed before these systems can be reliably deployed.

AI vision systems make at least eight distinct types of errors when describing images. The researchers categorized hallucinations into specific failure modes including misidentifying people's gender, making up geographical locations, misreading text, getting numbers wrong, and confusing objects based on context—suggesting that different remedies may be needed for different error types.

The problem is widespread across multiple AI systems. The study tested eight different Vision-Language models and found hallucination issues across all of them in both image captioning and question-answering tasks, indicating this is a systemic problem rather than isolated to one company's technology.

A new public dataset can help researchers tackle the problem. The researchers created VHILT, a dataset of 2,000 examples with human-verified annotations of different hallucination types, providing a shared resource that can help the AI research community develop better solutions and measure progress.

Hallucination in vision AI deserves as much attention as language AI. While most hallucination research has focused on text-based language models, this paper demonstrates that image-understanding systems have equally serious problems that warrant dedicated research and remediation efforts.