ExplainabilityTransparencyAccountabilitycs.CV
Towards Gradient-based Time-Series Explanations through a SpatioTemporal Attention Network
Min Hun Lee
May 18, 2024
arXiv.org
5 citations
Key Takeaways
-
Attention networks can help explain AI decisions about videos: The STAN model successfully identified which frames (moments) in videos were most important for making predictions, suggesting that transformer-based attention mechanisms are a viable approach for understanding how AI systems analyze temporal data.
-
Medical activity recognition benefits from explainability: When tested on medically relevant activities, the model demonstrated potential to pinpoint critical moments in videos, which could help healthcare professionals understand and trust AI systems used in clinical settings.
-
Gradient-based explanations work for time-series data: The research shows that saliency maps—a technique that highlights important parts of data—can be effectively applied to video and time-series analysis, making it easier to see why an AI model made a particular decision rather than treating it as a "black box."
-
Weak supervision enables practical AI training: The model was trained using only activity type labels rather than frame-by-frame annotations, suggesting that explainable AI systems can be developed without requiring expensive, detailed manual labeling of every moment in a video.
Abstract
In this paper, we explore the feasibility of using a transformer-based, spatiotemporal attention network (STAN) for gradient-based time-series explanations. First, we trained the STAN model for video classifications using the global and local views of data and weakly supervised labels on time-series data (i.e. the type of an activity). We then leveraged a gradient-based XAI technique (e.g. saliency map) to identify salient frames of time-series data. According to the experiments using the datasets of four medically relevant activities, the STAN model demonstrated its potential to identify important frames of videos.