Evaluating saliency methods on artificial data with different background types

Explainability methods produce inconsistent results: The study found that "heatmaps vary strongly between saliency methods and backgrounds," meaning different explanation techniques can give very different answers about why an AI model made a decision—even when analyzing the same image.

Current XAI methods lack rigorous testing: The researchers note that explainable AI approaches "have not always been objectively evaluated," highlighting a significant gap in how these tools are validated before being used to make important decisions.

Background context matters more than expected: The framework revealed that the type of background in an image substantially affects how saliency methods perform, suggesting that explanations may be less reliable when applied to new types of data or real-world scenarios.

Clinical and safety-critical applications need caution: The authors "strongly encourage further evaluation of saliency maps and xAI methods using this framework before applying these in clinical or other safety-critical settings," warning against deploying these explanation tools in high-stakes domains like healthcare without better validation.

A new testing framework is available: The researchers developed an objective evaluation method using synthetic data with known ground truth, providing a practical tool for organizations to test whether explanation methods are actually reliable before deployment.