An Experimental Investigation into the Evaluation of Explainability Methods

Many explainability evaluation metrics may be redundant: The study found that 14 different metrics used to evaluate XAI methods produced highly correlated results, suggesting that researchers and practitioners may not need to use all of them—some metrics essentially measure the same thing.

Small technical changes dramatically affect evaluation results: The research demonstrated that varying the "baseline hyperparameter" (a technical setting in how explanations are evaluated) significantly changed metric values, meaning the choice of evaluation approach can substantially alter which XAI method appears best, even when the actual explanation hasn't changed.

Current metrics struggle to distinguish good explanations from random ones: When the researchers tested evaluation metrics against dummy methods (like random saliency maps), they found limitations in how reliably these metrics could rank and identify truly useful explanations versus meaningless ones, raising concerns about whether we're accurately measuring explanation quality.

The field lacks standardized evaluation approaches: The paper highlights that while many XAI methods exist to help users understand AI decisions, there is no consensus on how to properly evaluate whether these explanations actually work—creating confusion about which methods are genuinely better.

Evaluation methodology itself needs scrutiny: This work suggests that before organizations adopt specific XAI methods, they should carefully examine not just the methods themselves, but also how those methods are being evaluated, as the evaluation process can be unreliable or misleading.