ExplainabilityTransparencyAccountabilitycs.LG
A general-purpose method for applying Explainable AI for Anomaly Detection
John Sipple, Abdou Youssef
July 23, 2022
International Syposium on Methodologies for Intelligent Systems
23 citations
2 influential
Key Takeaways
-
Explainability and interpretability serve different purposes: The paper distinguishes between explainability as an algorithmic task (identifying why an AI made a decision) and interpretability as a cognitive task (helping humans understand that explanation), suggesting that effective AI systems need to address both dimensions rather than treating them as interchangeable.
-
Anomaly detection needs better explanation methods: While explainability research has focused heavily on supervised learning (where AI learns from labeled examples), this paper addresses the less-studied challenge of explaining unsupervised anomaly detection systems—the type of AI often used to flag fraud, security threats, or equipment failures without predefined categories.
-
A new metric for measuring explanation quality: The paper introduces "Attribution Error" as a measurable way to evaluate whether an AI's explanation of an anomaly is actually accurate, demonstrating that their method based on Integrated Gradients produces significantly lower attribution errors than alternative approaches—providing a concrete standard for assessing explanation reliability.
-
Cognitive science should inform AI design: By drawing on insights from cognitive sciences, the paper suggests that responsible AI development requires understanding not just the technical aspects of how AI works, but also how humans actually process and trust explanations, making AI systems more genuinely useful rather than just technically sound.