To trust or not to trust an explanation: using LEAF to evaluate local linear XAI methods

Popular AI explanation methods have serious reliability problems: The paper reveals that widely-used explanation techniques like LIME and SHAP—considered state-of-the-art in the field—produce unstable explanations that can change unpredictably and sometimes explain the wrong prediction entirely, raising concerns about their trustworthiness in real-world applications.

There's no standard way to evaluate if explanations are actually good: The research highlights a critical gap in the XAI field: while experts agree explanations should have certain desirable qualities, there is "no consensus on how to quantitatively evaluate explanations in practice," meaning organizations currently lack objective benchmarks for assessing explanation quality.

Explanations often don't match what the methods claim to do: The paper documents a significant gap between the theoretical promises of explanation methods and their actual implementations, suggesting that even when developers follow published guidelines, the resulting explanations may not work as intended.

LEAF provides a standardized testing framework for explanations: The authors introduce an open-source tool that offers consistent metrics for evaluating local linear explanations, giving organizations and researchers a common reference point to assess whether explanations can be trusted before using them to support real decisions.

Explanations should be actively tested before deployment: The paper emphasizes that explanations shouldn't just be used to inspect models after the fact—they should be rigorously evaluated and validated as decision-support tools before being deployed in high-stakes applications.