Robustness of Explainable Artificial Intelligence in Industrial Process Modelling

XAI methods vary significantly in reliability: The study tested four popular explanation methods (SHAP, LIME, ALE, and Smooth Gradients) on an industrial furnace model and found they differ substantially in their ability to correctly identify which factors actually influence the process—meaning users cannot assume all explanation methods are equally trustworthy.

More accurate AI models don't automatically mean better explanations: The research found that a machine learning model's ability to predict outcomes accurately is "coupled with the correctness of the explainability," suggesting that a model making good predictions may still provide misleading explanations about why it makes those predictions.

Ground truth validation is essential for responsible AI deployment: The authors developed a novel scoring methodology comparing XAI outputs against known ground truth from simulations, highlighting that organizations should validate explanation methods against objective standards before relying on them for critical industrial decisions.

Explanation robustness degrades under noisy conditions: When the researchers added simulated noise to test data, the XAI methods' performance varied across a range of conditions, indicating that explanations may become unreliable when real-world data is messy or incomplete—a common scenario in industrial settings.

One-size-fits-all XAI approaches are insufficient: Different explanation methods showed different strengths and weaknesses, suggesting that responsible AI implementation requires careful selection and testing of specific XAI methods for particular applications rather than adopting generic solutions.