ExplainabilityTransparencyAccountabilitycs.AI
Eliminating The Impossible, Whatever Remains Must Be True
Jinqiang Yu, Alexey Ignatiev, Peter J. Stuckey, Nina Narodytska, Joao Marques-Silva
June 20, 2022
AAAI Conference on Artificial Intelligence
38 citations
2 influential
Key Takeaways
-
Current AI explanations may be unnecessarily complicated: Existing formal explanation methods assume features work independently, which can make their "why" explanations longer than needed—potentially making them harder for humans to understand and act upon.
-
AI explanations about what didn't happen are often unreliable: The paper identifies a specific problem with "why not" explanations (counterfactual explanations): they may suggest unrealistic alternative scenarios because they don't account for how features actually relate to each other in the real world.
-
Background knowledge improves both clarity and accuracy: By incorporating real-world context about how data actually behaves, the researchers show that explanations become both more concise and more trustworthy—addressing two key barriers to responsible AI deployment.
-
Transparency about explanation methods matters: The paper emphasizes that systems should report which background information was used to generate an explanation, allowing humans to verify whether the reasoning is sound rather than simply trusting the system's output.
-
Practical extraction of background knowledge is feasible: Rather than requiring manual specification, the researchers demonstrate that existing data analysis techniques can automatically extract meaningful patterns from datasets, making this approach practical for real-world AI systems.