TrustworthinessTransparencySafetycs.HC
Impact of Human-AI Interaction on User Trust and Reliance in AI-Assisted Qualitative Coding
Jie Gao, Junming Cao, ShunYi Yeo, Kenny Tsu Wei Choo, Zheng Zhang, Toby Jia-Jun Li, Shengdong Zhao, Simon Tangi Perrault
September 25, 2023
arXiv.org
4 citations
Key Takeaways
-
AI assistance works differently depending on the task complexity: The research found that qualitative coding isn't one uniform task—it involves multiple subtasks of varying difficulty. This means AI systems need customized approaches for different parts of the work rather than a one-size-fits-all solution, suggesting that responsible AI design requires understanding the specific context where it's being applied.
-
What people say about AI trust doesn't match how they actually use it: The study revealed a significant gap between participants' stated trust in the AI system and their actual reliance on it in practice. This finding highlights an important challenge for responsible AI: we cannot assume user surveys or self-reported comfort levels accurately reflect how people will actually interact with AI systems in real work.
-
Both over-reliance and under-reliance on AI pose risks: The research emphasized that users can become either too dependent on AI assistance or too skeptical of it—both problematic outcomes. This suggests that responsible AI systems need to be designed to encourage appropriate, calibrated reliance rather than simply maximizing user trust.
-
Human-AI interaction design significantly influences trustworthiness: The varying coding strategies tested (different text selection and code length approaches) produced different trust and reliance outcomes, indicating that how an AI system is designed to interact with humans directly shapes whether users will trust and use it responsibly.