Towards Understanding the Influence of Training Samples on Explanations

Training data directly shapes AI explanations: The paper demonstrates that individual training samples significantly influence not just what an AI model predicts, but how it explains its decisions. This means that to understand why an AI system provides a particular explanation, we need to examine the specific training data that shaped it.

Noisy or misleading training examples can distort explanations: The research highlights that problematic training data—including noisy or misleading examples—can affect a model's internal reasoning and the explanations it generates. This is particularly important for applications like recourse (helping users understand how to change outcomes), where poor explanations could lead people astray.

New methods can identify which training samples cause problematic explanations: The paper proposes an algorithm to trace back and identify which specific training examples are responsible for generating particular explanations. This capability is crucial for responsible AI, as it allows practitioners to audit and potentially remove or correct problematic training data when explanations are unreliable.

Data quality matters as much as model performance: Traditional AI evaluation focuses on whether models make accurate predictions, but this work shows we should equally care about whether the reasoning behind those predictions is sound. A model might perform well overall while still providing misleading explanations due to specific training samples.

Explainability requires transparency about training data: To truly trust AI explanations, organizations need visibility into how their training data influences those explanations—not just the final predictions. This adds another layer of responsibility for AI developers to document and validate their training datasets.