Matching Representations of Explainable Artificial Intelligence and Eye Gaze for Human-Machine Interaction

AI explanations can align with human attention: The study found that when AI systems explain their decisions using visual heatmaps (specifically layer-wise relevance propagation), these explanations increasingly match where human drivers actually look, suggesting the machine and human are focusing on similar task-relevant information.

Task-specific training improves human-machine alignment: The research demonstrates that neural networks trained for specific driving tasks produce explanations that correlate more closely with human eye gaze patterns, indicating that well-designed AI systems naturally attend to what humans consider important.

Visual explanations could guide human attention: By showing drivers where the AI system is focusing, these aligned heatmap explanations could potentially be used to direct human attention toward relevant areas of the road, offering a practical safety application for human-machine teaming in driving scenarios.

Explainability enables non-verbal communication: The paper proposes that matching AI explanations with human visual attention creates a foundation for rapid, non-verbal communication between humans and machines—important for time-sensitive situations like driving where verbal communication isn't practical.

Understanding both perspectives is essential: The research emphasizes that effective human-machine interaction requires bridging two different "languages"—how machines represent information through AI explanations and how humans process visual information through eye gaze—rather than forcing one to adapt to the other.