Seeing through the Eyes of AI: Situated Explainability in Augmented Reality

AR makes AI explanations more intuitive by showing them where they matter: Rather than forcing users to look at separate 2D screens and mentally map explanations back to real objects, the paper demonstrates that displaying AI reasoning directly in the physical environment through AR helps users understand decisions in context—addressing a key gap in how explainability is currently presented.

Real-time, spatial explanations reduce cognitive burden: By overlaying explanation information directly onto the objects and locations an AI system is analyzing, users no longer need to "decouple data exploration and capture from AI model interpretation," making it easier to follow the AI's reasoning as it happens in their actual workspace.

Existing explainability methods can work in AR environments: The research shows that known explainability techniques aren't limited to 2D displays—they can be adapted and applied in augmented reality settings, suggesting a practical pathway for making AI systems more transparent across different interaction contexts.

This approach is particularly valuable for AI systems operating in physical spaces: The paper identifies that AI systems working in real-world environments (rather than purely digital ones) benefit most from situated, spatial explanations, highlighting where this technology could have the greatest impact on user understanding and trust.