ExplainabilityTransparencyTrustworthinesscs.AI
Activation-Deactivation: A General Framework for Robust Post-hoc Explainable AI
Akchunya Chanchal, David A. Kelly, Hana Chockler
October 1, 2025
arXiv.org
3 citations
Key Takeaways
-
Addresses a fundamental problem with current explanation methods: The paper identifies that popular perturbation-based explainability techniques (which explain AI decisions by testing what happens when inputs are altered) rely on unrealistic, out-of-distribution modifications, potentially producing misleading explanations about how models actually work.
-
Proposes a practical alternative approach: Rather than artificially modifying input data, the Activation-Deactivation method "switches off" internal parts of the AI model itself to understand which components drive specific predictions, avoiding the need to create fake or unrealistic data scenarios.
-
Designed for easy real-world implementation: ConvAD, the implementation for image-recognition AI systems, works as a "drop-in mechanism" that can be added to already-trained models without retraining, making it practical for organizations to adopt without significant computational costs or disruption.
-
Demonstrates improved reliability and generalization: The paper shows ConvAD produces explanations that are both more robust (consistent and reliable) and more transferable (work better across different models and datasets) compared to existing state-of-the-art methods, suggesting more trustworthy insights into model behavior.
-
Validated across diverse conditions: The evaluation spans multiple neural network architectures, datasets, and perturbation strategies, indicating the findings are broadly applicable rather than limited to specific scenarios.