ExplainabilityTransparencyTrustworthinesscs.LG
Reliable Graph Neural Network Explanations Through Adversarial Training
Donald Loveland, Shusen Liu, Bhavya Kailkhura, Anna Hiszpanski, Yong Han
June 25, 2021
arXiv.org
5 citations
Key Takeaways
-
Post-hoc explanation methods for AI systems can be unreliable: The paper demonstrates that common approaches to explaining how graph neural networks make decisions often fail to accurately capture what the model actually learned, raising concerns about trusting these explanations for high-stakes applications.
-
Training methods matter for explainability: By using adversarial training during model development (similar to techniques already proven effective in computer vision), the researchers show that GNNs can be built to produce more trustworthy and interpretable explanations, suggesting that explainability should be considered during model creation rather than only after.
-
New evaluation tools are needed for explanation quality: The paper introduces a new metric to assess how well explanation methods actually reflect a model's learned patterns, particularly useful when ground truth data isn't available—addressing a practical gap in how we currently validate AI explanations.
-
Real-world benefits in chemistry: The adversarial training approach successfully helped extract more domain-relevant insights in chemistry applications, demonstrating that improved explainability can have tangible benefits for specialized fields where understanding model reasoning is critical.
Abstract
Graph neural network (GNN) explanations have largely been facilitated through post-hoc introspection. While this has been deemed successful, many post-hoc explanation methods have been shown to fail in capturing a model's learned representation. Due to this problem, it is worthwhile to consider how one might train a model so that it is more amenable to post-hoc analysis. Given the success of adversarial training in the computer vision domain to train models with more reliable representations, we propose a similar training paradigm for GNNs and analyze the respective impact on a model's explanations. In instances without ground truth labels, we also determine how well an explanation method is utilizing a model's learned representation through a new metric and demonstrate adversarial training can help better extract domain-relevant insights in chemistry.