ExplainabilityTransparencyAccountabilitycs.HC
Conceptualizing the Relationship between AI Explanations and User Agency
Iyadunni Adenuga, Jonathan Dodge
December 5, 2023
arXiv.org
5 citations
Key Takeaways
-
Explanations alone are insufficient for user agency: The paper finds that XAI systems must go beyond simply explaining AI decisions; explanations serve as "one of several possible first steps" for agency, suggesting that additional mechanisms are needed to help users actually act on what they learn.
-
User control is essential, especially for non-experts: The research observes that laypersons—particularly those the authors call "tinkerers" who want to experiment with systems—benefit most when explanations are combined with direct user control, enabling them to "make meaningful changes" rather than just understand decisions.
-
Agency requires converting understanding into future action: The paper emphasizes that true agency means allowing users to "convert forethought to outcome in a more effective manner in future interactions," indicating that responsible XAI should empower users to improve their own outcomes over time, not just comprehend past decisions.
-
One-size-fits-all explanations miss the mark: By adopting a "user-centric lens" through case studies, the research suggests that different users have different needs and capabilities, implying that responsible AI systems should be designed with specific user groups in mind rather than generic explanations for everyone.
Abstract
We grapple with the question: How, for whom and why should explainable artificial intelligence (XAI) aim to support the user goal of agency? In particular, we analyze the relationship between agency and explanations through a user-centric lens through case studies and thought experiments. We find that explanation serves as one of several possible first steps for agency by allowing the user convert forethought to outcome in a more effective manner in future interactions. Also, we observe that XAI systems might better cater to laypersons, particularly "tinkerers", when combining explanations and user control, so they can make meaningful changes.