AI2030 Responsible AI Tools Marketplace

Key Takeaways

Explanations can undermine user trust when inconsistent across similar cases: The paper demonstrates that AI systems often provide explanations that seem reasonable for individual predictions but contradict each other when users interact with the system multiple times, potentially causing users to lose confidence in the system even if its predictions are accurate.
Treating explanations as logical commitments reveals hidden problems: By formally modeling explanations as statements that commit the system to certain behaviors, researchers can identify and analyze inconsistencies that wouldn't be apparent from looking at single explanations in isolation—suggesting that explanations should be evaluated as part of ongoing interactions, not just one at a time.
Non-monotonic reasoning offers solutions to explanation inconsistencies: The paper proposes that explanation consistency can be improved by applying principles from non-monotonic reasoning (reasoning that can be revised with new information), such as using specificity relations to determine which explanations should take priority in different contexts.
Interactive explanation scenarios require new evaluation frameworks: Current methods for evaluating explanations focus on individual instances, but this work argues that responsible AI systems need evaluation approaches that consider how explanations behave across multiple user interactions and whether they maintain logical consistency over time.

Abstract

Recent work shows issues of consistency with explanations, with methods generating local explanations that seem reasonable instance-wise, but that are inconsistent across instances. This suggests not only that instance-wise explanations can be unreliable, but mainly that, when interacting with a system via multiple inputs, a user may actually lose confidence in the system. To better analyse this issue, in this work we treat explanations as objects that can be subject to reasoning and present a formal model of the interactive scenario between user and system, via sequences of inputs, outputs, and explanations. We argue that explanations can be thought of as committing to some model behaviour (even if only prima facie), suggesting a form of entailment, which, we argue, should be thought of as non-monotonic. This allows: 1) to solve some considered inconsistencies in explanation, such as via a specificity relation; 2) to consider properties from the non-monotonic reasoning literature and discuss their desirability, gaining more insight on the interactive explanation scenario.

Corrected version for the XAI-IJCAI 2022 workshop, expands on the XLoKR-KR 2022 workshop

Key Takeaways

Explanations can undermine user trust when inconsistent across similar cases: The paper demonstrates that AI systems often provide explanations that seem reasonable for individual predictions but contradict each other when users interact with the system multiple times, potentially causing users to lose confidence in the system even if its predictions are accurate.
Treating explanations as logical commitments reveals hidden problems: By formally modeling explanations as statements that commit the system to certain behaviors, researchers can identify and analyze inconsistencies that wouldn't be apparent from looking at single explanations in isolation—suggesting that explanations should be evaluated as part of ongoing interactions, not just one at a time.
Non-monotonic reasoning offers solutions to explanation inconsistencies: The paper proposes that explanation consistency can be improved by applying principles from non-monotonic reasoning (reasoning that can be revised with new information), such as using specificity relations to determine which explanations should take priority in different contexts.
Interactive explanation scenarios require new evaluation frameworks: Current methods for evaluating explanations focus on individual instances, but this work argues that responsible AI systems need evaluation approaches that consider how explanations behave across multiple user interactions and whether they maintain logical consistency over time.

Abstract

Corrected version for the XAI-IJCAI 2022 workshop, expands on the XLoKR-KR 2022 workshop

On Interactive Explanations as Non-Monotonic Reasoning

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links

On Interactive Explanations as Non-Monotonic Reasoning

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links