Human Agency, Causality, and the Human Computer Interface in High-Stakes Artificial Intelligence

Human agency, not trust, is the core problem: The paper argues that current focus on "trustworthy AI" misses the fundamental issue—high-stakes AI systems are eroding users' ability to understand and control outcomes. When users cannot perceive how a system reaches decisions, they lose causal control even if the system is technically reliable.

AI interfaces create a "double uncertainty" problem: Users face uncertainty about what the AI system will do, while the AI itself operates on probabilistic uncertainty. Current interfaces fail to adequately represent both layers of uncertainty, leaving users unable to make informed decisions or catch errors before they cause harm.

Explainable AI (XAI) currently falls short: The paper critiques existing XAI approaches for focusing on correlations rather than causality and for not properly communicating uncertainty to users. This means explanations may appear clear while actually obscuring how decisions are made.

"Bad AI functions as bad UI": Drawing on interface design principles, the paper frames catastrophic AI failures as fundamentally interface failures—systems that misrepresent their own state and capabilities, leading users to make mistakes they wouldn't otherwise make.

A Causal-Agency Framework is needed: The authors propose integrating causal models, uncertainty quantification, and human-centered evaluation into AI system design to restore users' ability to understand causality and maintain meaningful control over high-stakes decisions.