Geometric Remove-and-Retrain (GOAR): Coordinate-Invariant eXplainable AI Assessment

Current methods for testing AI explanations have a blind spot: The widely-used Remove-and-Retrain (ROAR) approach, which evaluates AI explanations by removing important pixels and retraining models, fails to distinguish between different explanation methods, meaning it cannot reliably tell us which explanation technique actually works better.

The problem stems from how we think about image features: When researchers looked at this issue from a geometric perspective—essentially examining the mathematical structure of how pixels relate to each other—they discovered that pixel-by-pixel removal strategies miss important patterns, compromising the validity of current XAI assessments.

A new evaluation method shows promise: The researchers developed Geometric Remove-and-Retrain (GOAR) as an alternative approach that overcomes these limitations, and demonstrated through experiments on both synthetic and real datasets that it provides more reliable evaluation of explanation methods than existing techniques.

This matters for trustworthy AI development: Since identifying which features influence AI decisions is "indispensable" for explainable AI, having unreliable evaluation methods means organizations may unknowingly deploy explanation systems that don't actually work as intended, undermining efforts to make AI more transparent and trustworthy.