Concerning Uncertainty -- A Systematic Survey of Uncertainty-Aware XAI

AI explanations often hide uncertainty, which can mislead users: The paper finds that most explainable AI systems don't communicate how confident they are in their explanations. This is problematic because a user might trust an explanation that the system is actually uncertain about, potentially leading to poor decisions.

Three main technical approaches exist for adding uncertainty to explanations, but there's no consensus on which works best: The survey identifies Bayesian methods, Monte Carlo approaches, and Conformal methods as the primary ways researchers incorporate uncertainty into AI explanations. However, the field lacks agreement on evaluation standards, making it difficult to compare which approach actually helps users make better decisions.

Current evaluation methods focus too much on the AI system and not enough on real people: The paper notes that "evaluation practices remain fragmented and largely model centered, with limited attention to users." This means researchers are often measuring whether uncertainty estimates are mathematically accurate rather than whether they actually help humans understand and trust AI systems appropriately.

Calibration and counterfactual explanations show promise for responsible AI: The authors highlight "calibration" (ensuring stated confidence levels match actual accuracy) and "counterfactual approaches" (showing what would need to change for a different outcome) as particularly valuable for making AI systems more trustworthy and interpretable.

The field needs unified standards for testing uncertainty-aware explanations: The survey calls for evaluation principles that connect uncertainty communication to actual human decision-making, suggesting that progress requires moving beyond fragmented technical metrics toward holistic assessment of whether these systems genuinely improve responsible AI deployment.