Causal Explanations from the Geometric Properties of ReLU Neural Networks

Geometric approach to explainability preserves model performance: Unlike existing methods that create simplified "distilled" versions of neural networks (which significantly degrade performance), this paper's geometric approach explains decisions directly from the original model, maintaining its accuracy while providing interpretability.

ReLU networks can be understood as piecewise linear functions: The research demonstrates that ReLU neural networks divide input space into distinct regions defined by n-dimensional convex polytopes, where each region applies a single linear function—providing a mathematically grounded way to understand how these networks make decisions.

Addresses a critical gap in autonomous systems safety: The paper tackles a key challenge in responsible AI: current explainability methods cannot guarantee they accurately reflect the original model's decision-making, making it impossible to assure safety in autonomous systems. This geometric approach aims to bridge that gap.

Enables "why" and "why not" explanations: By leveraging the geometric properties of ReLU networks, this method can generate causal explanations that answer both why a model made a particular decision and why it didn't make alternative decisions—crucial for understanding and auditing AI system behavior.

Applicable to real-world autonomous systems: The research specifically targets neural networks used in control policies for autonomous systems, suggesting practical applications for improving transparency and safety assurance in deployed AI systems.