ExplainabilityTransparencyTrustworthinesscs.HC
Interaction as Explanation: A User Interaction-based Method for Explaining Image Classification Models
Hyeonggeun Yun
April 15, 2024
arXiv.org
1 citations
Key Takeaways
-
Interactive explanation makes AI more accessible to non-experts: Traditional explainability methods are designed for specialists, but this paper demonstrates that allowing users to directly modify images (by painting and erasing parts) and observe how predictions change provides a more intuitive way for general users to understand how image classification models work.
-
Users can discover which features matter most through hands-on experimentation: By interacting with images in a web-based tool, users can identify which visual elements are critical to the model's decisions—essentially learning the model's logic through trial and error rather than studying technical visualizations.
-
Aligning user understanding with actual model behavior is crucial: The paper emphasizes that explainability should help users develop accurate "mental models" that match how the AI actually works, reducing the risk of misplaced trust or misuse based on incorrect assumptions about the system's reasoning.
-
Explainability design should prioritize end-user engagement: Rather than focusing solely on technical accuracy of explanations, this work highlights the importance of designing explainability methods that actively engage users and make the learning process intuitive and interactive.
Abstract
In computer vision, explainable AI (xAI) methods seek to mitigate the 'black-box' problem by making the decision-making process of deep learning models more interpretable and transparent. Traditional xAI methods concentrate on visualizing input features that influence model predictions, providing insights primarily suited for experts. In this work, we present an interaction-based xAI method that enhances user comprehension of image classification models through their interaction. Thus, we developed a web-based prototype allowing users to modify images via painting and erasing, thereby observing changes in classification results. Our approach enables users to discern critical features influencing the model's decision-making process, aligning their mental models with the model's logic. Experiments conducted with five images demonstrate the potential of the method to reveal feature importance through user interaction. Our work contributes a novel perspective to xAI by centering on end-user engagement and understanding, paving the way for more intuitive and accessible explainability in AI systems.