Disagreement amongst counterfactual explanations: How transparency can be deceptive

Counterfactual explanations can be manipulated to hide bias: The paper found "alarmingly high disagreement levels" between different explanation methods, which malicious actors could exploit to exclude sensitive features (like race or gender) from explanations, effectively "fairwashing" unfair AI models while appearing transparent.

Transparency tools can paradoxically be deceptive: Because multiple valid explanations exist for the same AI decision, stakeholders may be shown cherry-picked explanations that support a desired narrative rather than revealing the true factors driving the decision—undermining the core purpose of explainability.

The problem is widespread and empirically confirmed: Testing 12 different explanation methods across 40 datasets generated over 192,000 explanations, revealing that disagreement between methods is not a rare edge case but a systemic issue in how counterfactual explanations work.

Current regulations may be insufficient: As laws like the EU's right to explanation gain traction, this research shows that simply providing "an explanation" for high-stakes decisions (like loan denials or hiring) may not guarantee meaningful transparency if multiple conflicting explanations can be presented.

Standardization and governance are needed: The paper implies that without agreed-upon standards for which counterfactual explanation method to use, or safeguards against selective presentation, this XAI technique could become a tool for obscuring rather than revealing algorithmic decision-making.