On the Disagreement in Perturbation-based xAI -- Benchmarking Perturbation Choices for Flood Detection from SAR Images

Perturbation-based AI explanation methods are highly sensitive to technical choices: The study demonstrates that how researchers choose to alter and replace image regions significantly affects the explanations generated, meaning the same model can appear to use different reasoning depending on these parameter choices.

Different explanation methods can produce contradictory results for the same model: The research found that varying perturbation settings (patch size, shape, and replacement schemes) can yield "ambiguous and even contradictory explanations," raising concerns about the reliability of these explanations for understanding AI decision-making.

Technical parameters should be treated as critical methodological decisions: The authors emphasize that perturbation choices are "an integral part when interpreting explanations," suggesting that researchers and practitioners cannot treat these settings as minor technical details but must carefully document and justify them.

Current explanation methods may not reliably reflect how models actually reason: The study evaluated explanations for their "faithfulness to the model's reasoning," implying that some perturbation-based explanations may not accurately represent what the model is actually using to make decisions, which is crucial for trustworthy AI systems.

Responsible AI requires scrutiny beyond visual inspection of explanations: The research goes beyond simply looking at heatmaps to evaluate consistency and faithfulness, suggesting that stakeholders relying on AI explanations should demand rigorous validation rather than accepting explanations at face value.