Incremental Permutation Feature Importance (iPFI): Towards Online Explanations on Data Streams

Real-world AI systems need explanations that update continuously: Traditional explainability methods analyze static datasets, but many real-world AI systems learn from data streams (like fraud detection or medical monitoring). This paper addresses the gap by developing iPFI, an algorithm that explains which features matter most as new data arrives, rather than requiring complete retraining.

The proposed method works across different AI model types: iPFI is "model-agnostic," meaning it can explain decisions from any type of AI model without needing to redesign the explanation method for each one. This makes it practical for organizations using diverse AI systems.

Theoretical guarantees provide confidence in explanations: The authors prove mathematical guarantees about how accurate their incremental explanations are, specifically regarding the expected value and variance of feature importance scores. This theoretical foundation helps practitioners trust the explanations even as data continuously changes.

The algorithm handles concept drift—when the world changes: Real-world data streams often experience "concept drift," where patterns shift over time (e.g., customer behavior changing seasonally). The paper demonstrates that iPFI maintains reliable explanations under these dynamic conditions, not just in stable environments.

Efficiency gains enable real-time explanations: By computing feature importance incrementally rather than recalculating from scratch each time, iPFI reduces computational burden, making it feasible to provide timely explanations in fast-moving applications where batch reprocessing would be too slow.