A Study on Multimodal and Interactive Explanations for Visual Question Answering

Explanations genuinely help people understand AI failures: The study found that when a Visual Question Answering system gave wrong answers, users who received explanations were significantly better at predicting those errors than users without explanations, suggesting explanations are particularly valuable for catching AI mistakes.

User confidence in AI explanations correlates with actual understanding: The research discovered a strong correlation between how highly users rated explanations and their actual ability to predict AI system accuracy, indicating that people's intuitive sense of whether an explanation is good aligns with whether it actually improves their understanding.

Current evidence for XAI effectiveness has been limited: The authors highlight that while many explainable AI methods exist, there has been surprisingly little empirical evidence showing these explanations actually improve user trust, understanding, and usability—a gap this research begins to address through rigorous user testing.

Interactive explanations show promise for human-AI collaboration: The study demonstrates that multimodal explanations (combining different types of information) can improve how well humans and AI systems work together, suggesting that explanation design should be a priority for AI systems deployed in real-world settings where human judgment matters.

Measuring explanation quality requires testing with real users: Rather than assuming explanations work, the researchers used controlled experiments with human participants to evaluate effectiveness, highlighting that responsible AI development requires empirical validation of whether transparency measures actually achieve their intended goals.