Expressive Explanations of DNNs by Combining Concept Analysis with ILP

Visual explanations alone are insufficient for high-stakes applications: The paper identifies a critical limitation of standard visualization-based explanations—they cannot distinguish between different states of the same feature (e.g., whether an eye is open or closed) or accurately describe how the absence of features influences decisions, which is particularly problematic for regulated domains like medical AI and GDPR-compliant systems.

Symbolic rules offer more transparent decision-making: The researchers developed a method that converts learned neural network features into human-readable first-order logic rules, providing what they call "expressive, verbal explanations" that can better communicate the reasoning behind AI decisions to non-technical stakeholders and regulators.

Concept analysis bridges the gap between neural networks and human understanding: By training the system to recognize human-understandable visual concepts, the method enables the extraction of meaningful explanations from deep neural networks without requiring symbolic inputs upfront—addressing a major practical barrier to explainability in computer vision.

Faithfulness to the original model is achievable: The paper demonstrates that their symbolic explanations remain faithful to the original black-box model's decisions, suggesting that increased interpretability doesn't necessarily require sacrificing accuracy or representativeness.

Explainability is essential for regulatory compliance: The work emphasizes that transparency requirements in regulations like GDPR and the need for auditable decision-making in critical domains make advanced explanation methods not just desirable but necessary for responsible AI deployment.