Do RUL explanations hold up? Faithfulness and stability of attributions on C-MAPSS

Visual explanations from AI models aren't automatically trustworthy: The paper found that heatmaps showing which sensors or time periods matter most can "look mechanical" without actually being reliable guides for engineers—a key distinction between something that appears explanatory and something actionable.

Different explanation methods have significant weaknesses: When researchers tested three common attribution techniques (Integrated Gradients, occlusion, and Transformer attention), they found that Transformer attention maps were "weakly faithful," meaning removing the highlighted parts didn't actually change the model's predictions as much as the visualization suggested.

Stability across retraining is a separate problem from noise robustness: While all three explanation methods remained "almost unchanged under 5% input noise," Integrated Gradients and occlusion only showed "moderate agreement" when the model was retrained with different random seeds—revealing that explanations can be stable to small input changes but unstable to the randomness inherent in model training.

Occlusion and Integrated Gradients are more reliable for maintenance decisions: The paper recommends these two methods over attention-based explanations for real-world maintenance engineering reports, as they demonstrated stronger faithfulness in deletion tests (where removing highlighted features actually affected predictions).

Current AI explanations for equipment maintenance lack rigorous validation: The research highlights that the predictive maintenance field "still under-reports" standard XAI evaluation checks, meaning many deployed systems may be providing explanations to engineers without proper verification of their reliability.