ExplainabilityTransparencyTrustworthinesscs.LG
Prototypical Self-Explainable Models Without Re-training
Srishti Gautam, Ahcene Boubekki, Marina M. C. Höhne, Michael C. Kampffmeyer
December 13, 2023
Trans. Mach. Learn. Res.
6 citations
Key Takeaways
-
SEMs offer safety advantages over post-hoc methods: The paper identifies that self-explainable models (SEMs), which provide explanations directly during prediction, are "preferred in safety-critical scenarios" compared to post-hoc explanation methods that explain pre-trained black-box models after the fact—an important distinction for high-stakes applications like healthcare or autonomous systems.
-
Existing SEMs have adoption barriers: Current self-explainable models require "complex architectures and heavily regularized loss functions" with "specific and costly training," which has limited their real-world deployment despite their safety advantages. This technical and computational burden has kept post-hoc methods more popular despite being less transparent.
-
Explainability can be added without retraining: KMEx demonstrates that any existing pre-trained model can be converted into an explainable system using prototype-based explanations without requiring expensive retraining, potentially making transparent AI more accessible and practical for organizations with existing models.
-
Prototype-based explanations provide trustworthy transparency: The method uses "class-prototype-based explanations that are diverse and trustworthy," suggesting that showing users which training examples a model's decision resembles may be a more intuitive and reliable way to explain AI decisions than other explanation techniques.
-
Better evaluation methods are needed for explainable AI: The paper emphasizes the need for "more reliable and objective evaluation of SEMs," highlighting that the field currently lacks standardized ways to assess whether explanations are actually trustworthy and useful—a critical gap for responsible AI deployment.
Abstract
Explainable AI (XAI) has unfolded in two distinct research directions with, on the one hand, post-hoc methods that explain the predictions of a pre-trained black-box model and, on the other hand, self-explainable models (SEMs) which are trained directly to provide explanations alongside their predictions. While the latter is preferred in safety-critical scenarios, post-hoc approaches have received the majority of attention until now, owing to their simplicity and ability to explain base models without retraining. Current SEMs, instead, require complex architectures and heavily regularized loss functions, thus necessitating specific and costly training. To address this shortcoming and facilitate wider use of SEMs, we propose a simple yet efficient universal method called KMEx (K-Means Explainer), which can convert any existing pre-trained model into a prototypical SEM. The motivation behind KMEx is to enhance transparency in deep learning-based decision-making via class-prototype-based explanations that are diverse and trustworthy without retraining the base model. We compare models obtained from KMEx to state-of-the-art SEMs using an extensive qualitative evaluation to highlight the strengths and weaknesses of each model, further paving the way toward a more reliable and objective evaluation of SEMs (The code is available at https://github.com/SrishtiGautam/KMEx).
Paper Info
arXiv ID2312.07822
PublishedDecember 13, 2023
RevisedJune 4, 2024
Citations
6
Relevance82%
VenueTrans. Mach. Learn. Res.
TypeJournalArticle
Categorycs.LG
Responsible AI Topics
ExplainabilityTransparencyTrustworthiness
Links
Citation data from Semantic Scholar