Towards Self-Explainability of Deep Neural Networks with Heatmap Captioning and Large-Language Models

Heatmaps alone aren't enough for understanding AI decisions: While heatmaps are commonly used to show which parts of an image an AI system focuses on, the paper identifies that this visual approach has limitations in accessibility and scalability—meaning it's difficult to automatically explain what these heatmaps actually mean to different audiences.

Combining heatmaps with natural language makes AI explanations more accessible: The framework converts heatmap information into text descriptions using a template-based captioning approach, then uses large language models to generate human-readable explanations. This bridges the gap between technical visualization and understandable communication about how AI makes decisions.

Automated explanations could democratize AI interpretability: By automating the explanation process rather than requiring manual analysis of heatmaps, this approach makes it more feasible to explain AI decisions at scale—potentially helping organizations deploy more transparent AI systems without requiring specialized expertise for each decision.

Specialized knowledge improves AI explanations: The framework's reasoning module combines heatmap information with domain-specific knowledge through large language models, suggesting that better explanations require more than just visual attention data—they need contextual understanding of what the AI is actually looking at.