AI2030 Responsible AI Tools Marketplace

Key Takeaways

Explainability helps, but isn't a complete solution: The study of 562 participants found that providing explanations for AI recommendations does increase user compliance, but this effect is not uniform across all users, suggesting that transparency alone is insufficient for responsible AI deployment.
AI literacy matters as much as explanations: Users' existing knowledge about how AI works significantly affects whether they follow AI recommendations. This indicates that improving public understanding of AI is just as important as making individual AI systems more transparent.
Mental models are the missing link: The research reveals that the relationship between explanations, AI literacy, and compliance is mediated by users' mental models of AI—meaning how people conceptualize AI systems in their minds is a critical factor that designers need to consider and potentially influence.
Different explanation types have different effects: The study tested two different types of XAI explanations, finding that not all explanations are equally effective, which has practical implications for how organizations should design their AI transparency features rather than adopting a one-size-fits-all approach.
Designing for responsible AI requires a multi-faceted approach: Rather than focusing solely on making AI systems more explainable, the findings suggest that successful responsible AI implementation requires attention to user education, explanation design, and how users mentally conceptualize AI systems.

Abstract

AI is becoming increasingly common across different domains. However, as sophisticated AI-based systems are often black-boxed, rendering the decision-making logic opaque, users find it challenging to comply with their recommendations. Although researchers are investigating Explainable AI (XAI) to increase the transparency of the underlying machine learning models, it is unclear what types of explanations are effective and what other factors increase compliance. To better understand the interplay of these factors, we conducted an experiment with 562 participants who were presented with the recommendations of an AI and two different types of XAI. We find that users' compliance increases with the introduction of XAI but is also affected by AI literacy. We also find that the relationships between AI literacy XAI and users' compliance are mediated by the users' mental model of AI. Our study has several implications for successfully designing AI-based systems utilizing XAI.

Key Takeaways

Explainability helps, but isn't a complete solution: The study of 562 participants found that providing explanations for AI recommendations does increase user compliance, but this effect is not uniform across all users, suggesting that transparency alone is insufficient for responsible AI deployment.
AI literacy matters as much as explanations: Users' existing knowledge about how AI works significantly affects whether they follow AI recommendations. This indicates that improving public understanding of AI is just as important as making individual AI systems more transparent.
Mental models are the missing link: The research reveals that the relationship between explanations, AI literacy, and compliance is mediated by users' mental models of AI—meaning how people conceptualize AI systems in their minds is a critical factor that designers need to consider and potentially influence.
Different explanation types have different effects: The study tested two different types of XAI explanations, finding that not all explanations are equally effective, which has practical implications for how organizations should design their AI transparency features rather than adopting a one-size-fits-all approach.
Designing for responsible AI requires a multi-faceted approach: Rather than focusing solely on making AI systems more explainable, the findings suggest that successful responsible AI implementation requires attention to user education, explanation design, and how users mentally conceptualize AI systems.

Abstract

Investigating the Role of Explainability and AI Literacy in User Compliance

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links

Investigating the Role of Explainability and AI Literacy in User Compliance

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links