Do Metrics for Counterfactual Explanations Align with User Perception?

Current AI explanation metrics don't match what users actually find helpful: The study found that algorithmic metrics commonly used to evaluate counterfactual explanations (explanations showing how to change an input to get a different AI decision) have only weak correlations with human judgments of quality, meaning these automated measures may not capture what people actually need from explanations.

The problem varies significantly depending on context: Results showed that metric-human alignment was "strongly dataset-dependent," suggesting that metrics that work well for evaluating explanations in one domain may fail in another, making it difficult to establish universal standards for explanation quality.

Simply using more metrics doesn't solve the problem: Adding more algorithmic metrics to predictive models did not reliably improve predictions of human satisfaction, indicating there are fundamental structural limitations in how current metrics are designed rather than just needing more of them.

Human-centered evaluation is essential for trustworthy AI: Since explainability is considered critical for building trust in AI systems, the disconnect between automated metrics and actual user perception suggests that current evaluation approaches may be giving false confidence that AI systems are truly explainable and trustworthy.

The field needs to rethink how it measures explanation quality: The findings underscore that developing better AI explanations requires moving beyond purely algorithmic evaluation methods to incorporate direct human feedback and perception, rather than relying on automated metrics as proxies for user understanding.