On the Effectiveness of Methods and Metrics for Explainable AI in Remote Sensing Image Scene Classification

One-size-fits-all approaches don't work for remote sensing: Explainability methods developed for regular photos perform poorly when applied directly to satellite imagery, suggesting that AI systems analyzing Earth observation data need customized explanation tools rather than generic computer vision solutions.

Different explanation methods fail in different ways: Perturbation-based methods (Occlusion, LIME) are heavily influenced by how images are artificially modified during testing, gradient-based methods (GradCAM) struggle when multiple objects appear in the same image, and some relevance propagation methods (LRP) misrepresent how important different regions actually are.

Current evaluation metrics have serious blind spots: The metrics used to verify whether explanations are trustworthy inherit the same problems as the explanation methods themselves, meaning we may be validating flawed explanations as reliable—a critical issue for applications like environmental monitoring or disaster response where accuracy matters.

Remote sensing creates unique challenges: The spatial characteristics of satellite imagery—such as multiple overlapping land types or objects in a single scene—expose weaknesses in explanation methods that weren't apparent when these tools were tested only on standard photographs.

Stakeholders need transparency about limitations: Before deploying AI systems for remote sensing applications, organizations should understand that current explainability methods and their evaluation metrics have documented limitations, requiring careful validation rather than blind trust in automated explanations.