REVEX: A Unified Framework for Removal-Based Explainable Artificial Intelligence in Video

Video explanation methods need improvement in localization accuracy: Across all tested methods, localization-based metrics showed poor performance, with pointing game accuracy only reaching 53% and IoU-based metrics staying below 20%, suggesting current video explanation techniques struggle to precisely identify which parts of videos are most important to AI decisions.

Different explanation methods excel at different tasks: Video LIME performed best at deletion and insertion metrics (up to 31% lower deletion values, 30% higher insertion), while Video RISE excelled at average drop metrics (10% lower), indicating that no single explanation method works best for all purposes—organizations need to choose methods based on their specific needs.

Temporal information matters for video AI explanations: The framework adapted six existing computer vision techniques by adding temporal information and local explanations, suggesting that explaining AI decisions in videos requires accounting for how events unfold over time, not just analyzing individual frames.

Current XAI evaluation metrics have significant limitations: The paper highlights that existing metrics for evaluating explanations have varying suitability for different applications, meaning organizations should carefully consider which evaluation metrics align with their use case rather than relying on a single standard measure.

Systematic decomposition reveals method weaknesses: By breaking down explanations into steps, the research identified specific technical issues like problems with pixel clustering and input perturbations, demonstrating that rigorous testing can uncover where explanation methods fail and guide improvements.