ExplainabilityTransparencyTrustworthinesscs.AI
Sanity Checks Revisited: An Exploration to Repair the Model Parameter Randomisation Test
Anna Hedström, Leander Weber, Sebastian Lapuschkin, Marina MC Höhne
January 12, 2024
arXiv.org
12 citations
Key Takeaways
-
Explanation methods need better testing: The Model Parameter Randomisation Test (MPRT) is a key tool for checking whether AI explanation methods actually work, but recent research has found methodological problems with how it's being used in practice.
-
Two improved testing approaches: The paper proposes "Smooth MPRT" and "Efficient MPRT" as fixes—the first reduces noise in test results through better sampling, while the second avoids biased measurements by measuring how much an explanation's complexity increases when model parameters are randomized.
-
More reliable evaluation means more trustworthy AI: By improving how we test explanation methods, organizations can have greater confidence that the explanations their AI systems provide are genuinely reflecting what the model is actually doing, rather than producing misleading outputs.
-
Practical impact on XAI reliability: The experimental results show these proposed variants improve the reliability of explanation metrics, which is essential for responsible AI deployment where stakeholders need to understand and trust how AI systems make decisions.
Abstract
The Model Parameter Randomisation Test (MPRT) is widely acknowledged in the eXplainable Artificial Intelligence (XAI) community for its well-motivated evaluative principle: that the explanation function should be sensitive to changes in the parameters of the model function. However, recent works have identified several methodological caveats for the empirical interpretation of MPRT. To address these caveats, we introduce two adaptations to the original MPRT -- Smooth MPRT and Efficient MPRT, where the former minimises the impact that noise has on the evaluation results through sampling and the latter circumvents the need for biased similarity measurements by re-interpreting the test through the explanation's rise in complexity, after full parameter randomisation. Our experimental results demonstrate that these proposed variants lead to improved metric reliability, thus enabling a more trustworthy application of XAI methods.