Quantifying the Intrinsic Usefulness of Attributional Explanations for Graph Neural Networks with Artificial Simulatability Studies

Explanation quality is multidimensional: The paper demonstrates that "usefulness" (whether an explanation helps teach a concept) is a distinct and important dimension of explanation quality that differs from "faithfulness" (whether an explanation accurately represents how a model works), suggesting current evaluation methods may be incomplete.

Good explanations improve AI efficiency: The research finds that "relevant explanations can significantly boost the sample efficiency of graph neural networks," meaning AI systems trained with quality explanations require less data to learn effectively—a practical benefit beyond just understanding model decisions.

Testing explanations without human subjects is feasible: By using "explanation-supervisable graph neural networks" instead of costly human experiments, the authors demonstrate a scalable method to evaluate explanation quality, making it more practical for organizations to assess their AI systems' interpretability.

Explanations need robustness testing: The study analyzes "robustness towards noise and bias in the explanations," revealing that explanation quality can degrade when explanations contain errors or biases—an important consideration for real-world deployment where perfect explanations may be unavailable.

Graph AI explanations need specialized evaluation: The extension of simulatability studies to graph neural networks addresses a gap in responsible AI practices, as graph-based AI systems (used in recommendation systems, drug discovery, and social networks) previously lacked adequate explanation quality assessment tools.