ExplainabilityTransparencyTrustworthinesscs.CV
Greybox XAI: a Neural-Symbolic learning framework to produce interpretable predictions for image classification
Adrien Bennetot, Gianni Franchi, Javier Del Ser, Raja Chatila, Natalia Diaz-Rodriguez
September 26, 2022
Knowledge-Based Systems
35 citations
1 influential
Key Takeaways
-
Built-in explanations are more reliable than add-on methods: The paper argues that explanations should come directly from the model itself rather than being applied separately afterward, addressing a key problem where existing XAI methods may not accurately reflect how the model actually makes decisions.
-
A hybrid approach combining neural and symbolic AI improves trustworthiness: Greybox XAI uses both deep neural networks (for image processing) and transparent models like logistic regression (for clear decision-making), allowing the system to provide justifiable reasons for its predictions in critical applications.
-
The field needs formal standards for what "explanation" means: The authors emphasize that XAI lacks universal criteria for evaluating explanations, proposing mathematical axioms and definitions to establish clearer standards—important for ensuring consistency across different AI systems used in high-stakes decisions.
-
Transparency and accuracy don't have to be sacrificed for each other: By training the neural and symbolic components independently and then combining them, the framework maintains both the prediction power of deep learning and the interpretability needed for critical environments like healthcare or criminal justice.
Abstract
Although Deep Neural Networks (DNNs) have great generalization and prediction capabilities, their functioning does not allow a detailed explanation of their behavior. Opaque deep learning models are increasingly used to make important predictions in critical environments, and the danger is that they make and use predictions that cannot be justified or legitimized. Several eXplainable Artificial Intelligence (XAI) methods that separate explanations from machine learning models have emerged, but have shortcomings in faithfulness to the model actual functioning and robustness. As a result, there is a widespread agreement on the importance of endowing Deep Learning models with explanatory capabilities so that they can themselves provide an answer to why a particular prediction was made. First, we address the problem of the lack of universal criteria for XAI by formalizing what an explanation is. We also introduced a set of axioms and definitions to clarify XAI from a mathematical perspective. Finally, we present the Greybox XAI, a framework that composes a DNN and a transparent model thanks to the use of a symbolic Knowledge Base (KB). We extract a KB from the dataset and use it to train a transparent model (i.e., a logistic regression). An encoder-decoder architecture is trained on RGB images to produce an output similar to the KB used by the transparent model. Once the two models are trained independently, they are used compositionally to form an explainable predictive model. We show how this new architecture is accurate and explainable in several datasets.