Bridging the Gap: Gaze Events as Interpretable Concepts to Explain Deep Neural Sequence Models

Eye movements, not fixation points, drive AI decision-making: The research found that saccades (rapid eye movements between fixation points) are "substantially more important" than fixations (stable gazes) when deep learning models identify people based on eye-tracking data, suggesting AI systems prioritize dynamic movement patterns over where people look.

Peak velocity moments are critical: Within saccadic movements, the samples closest to peak velocity—the fastest point of the eye movement—were identified as "most influential" to the model's predictions, indicating that AI focuses on the most extreme moments of eye motion rather than the entire movement trajectory.

Quantitative analysis reveals hidden patterns: This work provides the first systematic, dataset-wide quantitative evaluation of which specific gaze events matter most to neural networks, moving beyond visual saliency maps to offer concrete measurements of concept influence—a methodological advance for understanding how AI interprets biometric data.

Physical eye movement properties affect AI interpretation: The study demonstrates that measurable characteristics like saccadic amplitude (how far the eye moves) and fixational dispersion (how steady a fixation is) meaningfully impact what the AI model considers important, showing that biological properties of eye movement directly shape algorithmic decision-making.

Interpretability requires domain-specific knowledge: By combining established eye-tracking science (gaze event detection) with AI explainability methods, the research shows that explaining AI systems effectively requires integrating expertise from the original domain—in this case, oculomotor science—rather than relying on generic explanation techniques alone.