Explainable Interface for Human-Autonomy Teaming: A Survey

Black-box AI systems pose risks in critical domains: Foundation models integrated into safety-critical applications like transportation, medical, and defense systems lack transparency, making it difficult for humans to understand why autonomous systems make specific decisions—a particular concern when human lives are at stake.

Explainable interfaces are distinct from explanations themselves: The paper clarifies that an "explainable interface" is not simply providing explanations; rather, it's the designed system through which humans interact with and understand AI decisions, requiring careful consideration of how information is presented to users.

Human-centered evaluation is essential, not just technical performance: Responsible AI deployment requires assessing not only whether a model performs well technically, but also measuring human-centered factors (like trust and understanding) and whether teams can effectively accomplish their tasks together—a more comprehensive approach than traditional AI evaluation.

Mutual understanding between humans and AI is foundational to safe teaming: The survey emphasizes that transparency challenges in human-autonomy teams can only be addressed by fostering genuine two-way understanding and trust, rather than simply making AI systems more "explainable" in isolation.

Designing for human-AI collaboration requires interdisciplinary expertise: Effective explainable interfaces draw insights from psychology, human-computer interaction, and XAI research, suggesting that responsible AI development cannot rely on technical solutions alone but must integrate social and behavioral sciences.