ExplainabilityTransparencyTrustworthinesscs.AI
FastRM: An efficient and automatic explainability framework for multimodal generative models
Gabriela Ben-Melech Stan, Estelle Aflalo, Man Luo, Shachar Rosenman, Tiep Le, Sayak Paul, Shao-Yen Tseng, Vasudev Lal
December 2, 2024
arXiv.org
2 citations
Key Takeaways
-
Faster explainability enables real-world deployment: FastRM reduces computation time by 99.8% and memory usage by 44.4% compared to traditional methods, making it practical to explain AI decisions in real-time rather than as a time-consuming afterthought.
-
Vision-language models need better transparency tools: Large multimodal AI systems that process both images and text can generate false information, and this paper addresses the critical need for methods that help identify when models are making ungrounded claims.
-
Explainability becomes scalable with efficient methods: By making relevancy maps (visual explanations of what the model "looked at") faster to generate, FastRM removes a major barrier to deploying explainability features in production systems where speed matters.
-
Users can better evaluate AI reliability: The framework provides both visual explanations and quantitative confidence scores, giving people concrete tools to assess whether they should trust a particular AI output before acting on it.
Abstract
Large Vision Language Models (LVLMs) have demonstrated remarkable reasoning capabilities over textual and visual inputs. However, these models remain prone to generating misinformation. Identifying and mitigating ungrounded responses is crucial for developing trustworthy AI. Traditional explainability methods such as gradient-based relevancy maps, offer insight into the decision process of models, but are often computationally expensive and unsuitable for real-time output validation. In this work, we introduce FastRM, an efficient method for predicting explainable Relevancy Maps of LVLMs. Furthermore, FastRM provides both quantitative and qualitative assessment of model confidence. Experimental results demonstrate that FastRM achieves a 99.8% reduction in computation time and a 44.4% reduction in memory footprint compared to traditional relevancy map generation. FastRM allows explainable AI to be more practical and scalable, thereby promoting its deployment in real-world applications and enabling users to more effectively evaluate the reliability of model outputs.