Qualitative Investigation in Explainable Artificial Intelligence: A Bit More Insight from Social Science

Many XAI user studies lack rigorous social science foundations: The paper finds that qualitative research in XAI often lacks underlying theoretical frameworks and systematic methodological approaches that are standard in social science disciplines, suggesting current studies may not be capturing user needs as effectively as possible.

Four key areas need improvement in XAI research: The authors identify critical gaps in how XAI researchers conduct user studies—specifically in establishing theoretical grounding, choosing appropriate methodological approaches, designing data collection methods (interviews, focus groups, questionnaires), and analyzing the data collected.

Collaboration with social scientists is essential: The paper reinforces existing calls within the XAI community for AI researchers to partner with experts from social disciplines like sociology, psychology, and anthropology to strengthen the quality and reliability of user studies that test whether AI explanations actually help people.

Current XAI evaluation may be missing important insights: Without rigorous qualitative methods, researchers risk misunderstanding what explanations users actually need or whether AI systems are truly becoming more trustworthy and understandable to the people who use them.

Better user studies lead to better AI systems: By adopting more rigorous social science approaches to studying how people interact with AI explanations, developers can create AI systems that are genuinely more transparent and aligned with what users actually need to understand and trust these systems.