A Quantitative Evaluation Framework for Temporal Explainability in Echocardiographic Video Segmentation

Temporal explanations in medical AI require new evaluation methods: The paper demonstrates that standard explanation techniques (like Grad-CAM) used to interpret AI decisions in medical videos don't adequately capture how explanations change over time, highlighting a gap in current explainability practices for temporal models.

Model architecture significantly affects explanation stability: ConvLSTM models showed substantially lower consistency in their explanations at intermediate processing stages compared to final predictions, with Temporal Bottleneck designs producing more stable explanations than Encoder Bottleneck designs—suggesting that how models are built affects not just accuracy but also the trustworthiness of their explanations.

Good performance doesn't guarantee trustworthy explanations: Despite comparable segmentation accuracy across different model architectures, the explanations for why models made decisions varied dramatically, indicating that clinicians cannot assume a well-performing model provides reliable insights into its reasoning.

Current explanation metrics are insufficient for medical video AI: Frame-by-frame explanation analysis cannot distinguish between meaningful changes in how the model processes information over time versus genuine instability in explanations—a critical distinction for clinical deployment where understanding model reasoning is essential.

Temporal-aware explainability methods are needed for medical AI: The findings call for new explanation techniques specifically designed for video-based medical models that account for how AI representations evolve across frames, rather than adapting image-based explanation methods.