The Utility of Explainable AI in Ad Hoc Human-Machine Teaming

Explainable AI helps novices but can hinder experts: The research found that less experienced users benefit from AI explanations by gaining better situational awareness, but experienced users actually perform worse when given these explanations—suggesting that experienced practitioners may find the additional information distracting rather than helpful.

Explainable AI is not universally beneficial: The study demonstrates that the effectiveness of xAI depends heavily on who is using it and their level of expertise, challenging the assumption that more transparency is always better. Organizations cannot simply deploy xAI solutions without considering their specific team composition.

Cognitive overhead is a real concern: Novice users showed susceptibility to cognitive overhead when using xAI, meaning that while explanations improved their understanding, they also created mental burden that could negatively impact performance—a critical consideration when designing AI systems for human teams.

Shared understanding matters for teamwork: The research highlights that xAI's primary value lies in helping humans develop a "shared mental model" with AI systems, which is especially important in ad hoc teams where humans and machines haven't worked together before and need to quickly understand each other's decision-making.

One-size-fits-all AI explanations won't work: The findings suggest that responsible AI deployment requires tailoring explanations to user expertise levels rather than providing uniform transparency across all users.