ExplainabilitySafetyTransparencycs.LG
"Even if ..." -- Diverse Semifactual Explanations of Reject
André Artelt, Barbara Hammer
July 5, 2022
IEEE Symposium Series on Computational Intelligence
17 citations
2 influential
Key Takeaways
-
Reject options are critical for safety: Machine learning systems used in high-stakes decisions (like medical diagnosis or loan approval) need the ability to say "I'm not confident enough to decide" rather than making uncertain predictions, and this capability is especially important in safety-critical applications.
-
Explaining rejections is underexplored: While AI explainability has advanced significantly, there is "very little prior work" on explaining why a system rejected a decision—a gap this research begins to address, suggesting this remains an underdeveloped area in responsible AI.
-
Semifactual explanations offer a new approach: The paper proposes using "semifactual explanations" (example-based methods showing "even if" scenarios) as a way to help users understand rejection decisions, representing a novel methodology that hasn't been widely adopted in the XAI community yet.
-
Practical implementation tested: The researchers demonstrated their approach on conformal prediction systems, a specific type of machine learning method, suggesting the framework can be applied to real-world reject options rather than remaining purely theoretical.
-
Transparency in uncertain decisions matters: By developing methods to explain why systems reject inputs, this work supports the principle that users deserve to understand not just what a system decides, but also when and why it chooses not to decide—crucial for building trust in AI systems.