Re-Visiting Explainable AI Evaluation Metrics to Identify The Most Informative Features

Current methods for evaluating AI explanations have significant blind spots: The paper demonstrates that two widely-used metrics—ROAR (RemOve And Retrain) and Permutation Importance—don't reliably identify which features are actually most important to a model's decisions, particularly when features are related to each other or when models have many important features.

Multicollinearity creates a major problem for AI transparency: When input features are correlated with each other (a common real-world scenario), existing evaluation metrics become unreliable, meaning organizations may incorrectly believe they understand which factors their AI systems are using to make decisions.

A new evaluation approach could improve trustworthiness: The paper proposes a new metric called Expected Accuracy Interval (EAI) that provides upper and lower bounds for model performance, offering a more realistic picture of which features truly matter—especially useful when dealing with correlated data.

Model characteristics affect explanation quality assessment: The reliability of current explanation metrics depends on factors like overall model accuracy and the number of significant features present, meaning the same evaluation method may work well for some AI systems but fail for others.

Better evaluation metrics are essential for responsible AI deployment: Since organizations rely on these metrics to understand and justify their AI systems' decisions to stakeholders and regulators, improving their accuracy is crucial for building trustworthy and accountable AI systems.