On Generating Plausible Counterfactual and Semi-Factual Explanations for Deep Learning

Counterfactual explanations are increasingly popular but underutilized for images: While counterfactual explanations (showing what would need to change for a different prediction) have gained traction in AI explainability, most existing methods focus on tabular data rather than images, leaving a significant gap in explaining visual AI systems like computer vision models.

"Semifactuals" offer a more human-like explanation approach that has been overlooked: The paper highlights that semifactuals—explanations showing minimal changes that still support the original prediction—mirror how humans naturally explain their reasoning, yet have received virtually no attention in AI research despite their potential value.

Plausibility is a critical but often-ignored challenge in generating explanations: A major barrier to useful AI explanations is ensuring that counterfactual images look realistic and belong to the actual data distribution; many existing methods generate explanations that appear artificial or implausible, which undermines their credibility and usefulness.

PIECE method successfully generates more realistic visual explanations: The paper's proposed method (PIECE) outperforms existing approaches by identifying and modifying only the "exceptional" features in an image that drive the AI's decision, resulting in counterfactuals and semifactuals that appear more plausible across multiple measures in controlled experiments.

Better explanations could strengthen legal and psychological acceptance of AI decisions: The paper notes that improved counterfactual explanations have potential benefits for legal compliance and human understanding of AI systems, suggesting that more plausible explanations may help build justified trust in AI decision-making.