On the Relationship Between Explanations, Fairness Perceptions, and Decisions

Explanations alone may not guarantee fairer decisions: While explanations are often proposed as a solution to help humans make fairer decisions when working with AI systems, the paper finds that research evidence does not conclusively show explanations actually achieve this goal in practice, suggesting a gap between theory and real-world outcomes.

The path from explanations to fairness is complex: The paper identifies that explanations influence fairness outcomes through multiple interconnected factors—including how fair people perceive the AI system to be, how much they rely on its recommendations, and whether they actually override it when needed—rather than working through a single straightforward mechanism.

Humans may not override AI recommendations even when they should: For human-AI collaboration to improve fairness, people need to both understand explanations and be willing to reject algorithmic recommendations when appropriate. The paper suggests this override behavior may not happen reliably, even with explanations present.

Fairness perceptions don't automatically translate to fair decisions: The research framework distinguishes between whether people think an AI system is fair (their perception) and whether their actual decisions become more fair—these are not necessarily the same thing, which has important implications for how we design and evaluate explanation systems.

Better research design is needed: The paper calls for more carefully designed experiments that measure the actual relationship between explanations, fairness perceptions, reliance, and real decision outcomes, rather than assuming explanations will automatically improve fairness.