Quantitative Evaluations on Saliency Methods: An Experimental Study

No single explanation method is universally reliable: The study tested multiple saliency methods (techniques that highlight which parts of an image a model focuses on) and found that "no single explanation method dominates others in all metrics," meaning practitioners cannot assume any one technique will work best across different situations.

Current XAI metrics lack standardization: The paper identifies that explainable AI "lacks rigorous definition and fair metrics," highlighting a critical gap in how the field evaluates whether AI explanations are actually trustworthy—a foundational problem for responsible AI deployment.

Grad-CAM and RISE are relatively strong performers: Among the methods tested, Gradient-weighted Class Activation Mapping (Grad-CAM) and Randomly Input Sampling for Explanation (RISE) "perform fairly well in most of the metrics," providing some guidance for organizations selecting explanation tools, though neither is perfect.

Multiple dimensions of explanation quality matter: The study evaluated explanations across five different criteria—faithfulness (accuracy), localization (precision), false-positives (incorrect highlights), sensitivity, and stability—suggesting that responsible AI requires checking explanations from multiple angles rather than relying on a single quality measure.

Important measurement gaps remain: The authors note that current metrics "miss" certain measuring factors, indicating that even this comprehensive evaluation doesn't capture everything needed to truly validate whether AI explanations are reliable for high-stakes decisions.