MEGL: Multimodal Explanation-Guided Learning

Combining visual and textual explanations improves AI transparency: The paper demonstrates that using both types of explanations together—visual highlights showing where the model looks and text explaining why—provides more complete and reliable insights than either explanation type alone, addressing a key gap in current explainable AI methods.

AI explanations are often incomplete or inconsistent: The research identifies that visual explanations can lack reasoning context while textual explanations lack spatial grounding, and both can be unreliable. This finding highlights why users shouldn't fully trust single-modality explanations from current AI systems.

AI models can learn to improve accuracy through better explanations: By training models to generate spatially grounded explanations (text linked to specific image regions), MEGL improved classification performance, suggesting that the process of explaining decisions helps AI systems make better decisions.

Explanations can be learned even without perfect training data: The framework successfully generates aligned visual and textual explanations even when complete ground truth annotations are unavailable, making explainable AI more practical for real-world applications where comprehensive labeled data is expensive or difficult to obtain.

Multimodal explanations enable better consistency checks: By aligning visual and textual explanations with each other and with broader dataset patterns, the approach creates internal consistency mechanisms that could help detect when an AI model's reasoning is unreliable or biased.