"Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interaction

Users want practical guidance, not technical details: Study participants using the Merlin bird identification app prioritized explanations that could help them work better with the AI system over technical descriptions of how the AI works internally. This suggests XAI design should focus on actionable information rather than algorithmic complexity.

Explainability serves multiple purposes beyond understanding: The 20 end-users studied intended to use explanations for four distinct purposes: calibrating their trust in the AI, improving their own skills at the task, adjusting their behavior to provide better inputs to the system, and offering feedback to developers. This indicates a single explanation may need to serve multiple user goals simultaneously.

Human-like explanations resonate most: Among existing XAI approaches tested, participants preferred part-based explanations that mirror how humans reason about decisions (for example, highlighting which features of a bird image led to an identification). This preference suggests XAI design should align with natural human communication patterns rather than purely technical visualization methods.

Explainability enables human-AI collaboration: Rather than viewing explanations as one-way information transfer, the research frames XAI as a tool for improving the working relationship between humans and AI systems. Users can learn to provide better inputs and give constructive feedback when they understand how the AI responds to different information.