ExplainabilityTransparencyAccountabilitycs.LG
XInsight: Revealing Model Insights for GNNs with Flow-based Explanations
Eli Laird, Ayesh Madushanka, Elfi Kraka, Corey Clark
June 7, 2023
xAI
2 citations
Key Takeaways
-
GNNs lack transparency in high-stakes applications: Graph neural networks are widely used in critical domains like drug discovery and medical diagnosis, but they function as "black boxes" where it's unclear what patterns the models actually learn, creating risks when errors go undetected.
-
XInsight generates multiple valid explanations rather than just one: Unlike previous explainability methods that identify only a single "best" explanation, XInsight produces a diverse distribution of explanations using GFlowNets, providing a more complete picture of model reasoning and reducing the risk of missing important patterns.
-
The method was validated on real drug discovery tasks: When tested on the MUTAG dataset (classifying mutagenic compounds), XInsight's explanations clustered by lipophilicity, a known chemical property linked to mutagenicity, demonstrating that the model's learned patterns align with established scientific knowledge.
-
Explainability enables human oversight and knowledge discovery: By making model decisions interpretable, XInsight allows domain experts to recognize when models make errors and potentially discover new scientific insights—essential for responsible deployment in fields like pharmaceutical development where human validation is critical.
Abstract
Progress in graph neural networks has grown rapidly in recent years, with many new developments in drug discovery, medical diagnosis, and recommender systems. While this progress is significant, many networks are `black boxes' with little understanding of the `what' exactly the network is learning. Many high-stakes applications, such as drug discovery, require human-intelligible explanations from the models so that users can recognize errors and discover new knowledge. Therefore, the development of explainable AI algorithms is essential for us to reap the benefits of AI. We propose an explainability algorithm for GNNs called eXplainable Insight (XInsight) that generates a distribution of model explanations using GFlowNets. Since GFlowNets generate objects with probabilities proportional to a reward, XInsight can generate a diverse set of explanations, compared to previous methods that only learn the maximum reward sample. We demonstrate XInsight by generating explanations for GNNs trained on two graph classification tasks: classifying mutagenic compounds with the MUTAG dataset and classifying acyclic graphs with a synthetic dataset that we have open-sourced. We show the utility of XInsight's explanations by analyzing the generated compounds using QSAR modeling, and we find that XInsight generates compounds that cluster by lipophilicity, a known correlate of mutagenicity. Our results show that XInsight generates a distribution of explanations that uncovers the underlying relationships demonstrated by the model. They also highlight the importance of generating a diverse set of explanations, as it enables us to discover hidden relationships in the model and provides valuable guidance for further analysis.