Uncertainty in Additive Feature Attribution methods

Explanation methods themselves are uncertain: The paper demonstrates that popular AI explanation tools like LIME don't always provide consistent explanations—they have built-in uncertainty that varies depending on the instance being explained, which is important for users to understand when relying on these explanations for decision-making.

Important features can be explained with more confidence: The researchers found that features identified as important by explanation algorithms don't necessarily have lower uncertainty. They propose a fix that adjusts how these algorithms sample data so that important features receive more reliable explanations without requiring extra computing power.

Some predictions are easier to explain than others: The paper identifies "stable instances"—cases where the model's behavior is straightforward and explanations are highly certain. Understanding which predictions fall into this category could help organizations know when they can trust explanations more confidently.

More complex AI models produce less reliable explanations: The study reveals a direct relationship between model complexity and explanation uncertainty—more sophisticated models generate explanations with greater inherent uncertainty, suggesting a potential trade-off between model performance and explainability that organizations should consider.

Complexity of AI systems should be measured and accounted for: The authors propose a new metric to quantify how complex a "black box" AI model is, which could be used to adjust explanation algorithms and help achieve more trustworthy and consistent explanations across different types of AI systems.