Formalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI

Trust in AI requires two critical conditions: Users must be vulnerable (dependent on the AI system) and able to anticipate how the AI's decisions will affect them. Without both elements, meaningful trust cannot form, suggesting that transparency about AI capabilities and limitations is essential for appropriate reliance on these systems.

"Warranted" vs. "unwarranted" trust matters significantly: The paper distinguishes between trust that is justified by an AI system's actual reliability and trustworthiness versus trust that isn't. This distinction is crucial because unwarranted trust can lead to misuse or over-reliance on AI, while insufficient warranted trust can result in people rejecting beneficial technologies.

Trust operates like an implicit contract: Rather than viewing AI trust as similar to interpersonal trust between humans, this work frames it as a contractual relationship—users trust that an AI system will fulfill certain implicit or explicit promises about its behavior and limitations. This framing helps clarify what users are actually relying on when they trust an AI system.

Trustworthiness can be designed and evaluated: The paper provides a framework for assessing whether an AI system is actually trustworthy and whether users' trust in it is warranted, moving beyond subjective impressions to more objective evaluation criteria. This enables developers to intentionally build trustworthy systems rather than leaving trust to chance.

Explainability (XAI) directly supports appropriate trust: By connecting trust formalization to explainable AI, the work suggests that making AI decisions understandable to users is fundamental to helping people develop warranted—rather than blind or excessive—trust in these systems.