ExplainabilityTrustworthinessTransparencycs.LG
REPEAT: Improving Uncertainty Estimation in Representation Learning Explainability
Kristoffer K. Wickstrøm, Thea Brüsch, Michael C. Kampffmeyer, Robert Jenssen
December 11, 2024
AAAI Conference on Artificial Intelligence
2 citations
Key Takeaways
-
Uncertainty matters for trustworthy AI explanations: The paper demonstrates that current methods for explaining AI decisions often fail to clearly indicate how confident the system is about which parts of an image are important—a critical gap for users who need to know when to trust an explanation.
-
REPEAT provides "certainty scores" for image explanations: Rather than just showing which pixels matter, the new REPEAT method quantifies how certain the AI is about each pixel's importance by treating pixels as either definitely important or definitely unimportant, giving users clearer confidence levels in the explanations.
-
Better detection of unusual data: The evaluation shows REPEAT is more effective at identifying out-of-distribution data (images that differ significantly from what the AI was trained on), which is important for flagging cases where explanations may be unreliable.
-
Simpler, more intuitive explanations: REPEAT produces more concise explanations compared to existing methods, potentially making AI decision-making more understandable to non-technical stakeholders who need to interpret and act on these explanations.
Abstract
Incorporating uncertainty is crucial to provide trustworthy explanations of deep learning models. Recent works have demonstrated how uncertainty modeling can be particularly important in the unsupervised field of representation learning explainable artificial intelligence (R-XAI). Current R-XAI methods provide uncertainty by measuring variability in the importance score. However, they fail to provide meaningful estimates of whether a pixel is certainly important or not. In this work, we propose a new R-XAI method called REPEAT that addresses the key question of whether or not a pixel is \textit{certainly} important. REPEAT leverages the stochasticity of current R-XAI methods to produce multiple estimates of importance, thus considering each pixel in an image as a Bernoulli random variable that is either important or unimportant. From these Bernoulli random variables we can directly estimate the importance of a pixel and its associated certainty, thus enabling users to determine certainty in pixel importance. Our extensive evaluation shows that REPEAT gives certainty estimates that are more intuitive, better at detecting out-of-distribution data, and more concise.