ExplainabilityTransparencyAccountabilitycs.LG
Viewing the process of generating counterfactuals as a source of knowledge: a new approach for explaining classifiers
Vincent Lemaire, Nathan Le Boudec, Victor Guyomard, Françoise Fessant
September 8, 2023
IEEE International Joint Conference on Neural Network
3 citations
Key Takeaways
-
Counterfactual explanations can serve dual purposes: Rather than just explaining individual predictions, the process of generating counterfactuals (simulating "what-if" scenarios) can create reusable knowledge about how a model works, which can be stored and applied to future decisions.
-
Knowledge from counterfactuals has practical value: By treating counterfactual generation as a knowledge-creation process, organizations can build up a repository of insights about their models' behavior that goes beyond one-time explanations of specific predictions.
-
Naive Bayes classifiers are particularly well-suited for this approach: The paper demonstrates that naive Bayes classifiers have "interesting properties" that make them especially effective for extracting and storing knowledge through counterfactual reasoning, making them a good choice when explainability and knowledge reuse are priorities.
-
This approach bridges explanation and model understanding: Rather than viewing explainability as merely answering "why did the model decide this?", this framework suggests it can also answer "what does the model know?" by systematically exploring how feature changes affect predictions.
Abstract
There are now many explainable AI methods for understanding the decisions of a machine learning model. Among these are those based on counterfactual reasoning, which involve simulating features changes and observing the impact on the prediction. This article proposes to view this simulation process as a source of creating a certain amount of knowledge that can be stored to be used, later, in different ways. This process is illustrated in the additive model and, more specifically, in the case of the naive Bayes classifier, whose interesting properties for this purpose are shown.