ExplainabilityTransparencyAccountabilitycs.LG
Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyond
Anna Hedström, Leander Weber, Dilyara Bareeva, Daniel Krakowczyk, Franz Motzkus, Wojciech Samek, Sebastian Lapuschkin, Marina M. -C. Höhne
February 14, 2022
Journal of machine learning research
317 citations
24 influential
Key Takeaways
-
Explanation methods need systematic evaluation: The paper identifies that while explainability is meant to build trust in AI systems, there has been limited research into actually verifying whether explanation methods are correct and reliable—highlighting a critical gap in responsible AI practices.
-
A new toolkit addresses reproducibility challenges: Quantus provides researchers with a standardized, open-source Python toolkit containing multiple evaluation metrics for explanation methods, helping ensure that AI explanations can be consistently tested and compared across different studies and organizations.
-
Transparency requires shared evaluation standards: By making Quantus publicly available and well-documented, the toolkit promotes reproducibility in explainability research, allowing different teams to evaluate explanation methods using the same rigorous benchmarks rather than ad-hoc approaches.
-
Explainability itself requires accountability: The paper emphasizes that since explanations are designed to increase trust in AI systems, those explanations themselves must be thoroughly evaluated and validated—suggesting that responsible AI requires multiple layers of verification, not just trusting that explanations are helpful.
Abstract
The evaluation of explanation methods is a research topic that has not yet been explored deeply, however, since explainability is supposed to strengthen trust in artificial intelligence, it is necessary to systematically review and compare explanation methods in order to confirm their correctness. Until now, no tool with focus on XAI evaluation exists that exhaustively and speedily allows researchers to evaluate the performance of explanations of neural network predictions. To increase transparency and reproducibility in the field, we therefore built Quantus -- a comprehensive, evaluation toolkit in Python that includes a growing, well-organised collection of evaluation metrics and tutorials for evaluating explainable methods. The toolkit has been thoroughly tested and is available under an open-source license on PyPi (or on https://github.com/understandable-machine-intelligence-lab/Quantus/).