Explainability for Fault Detection System in Chemical Processes

XAI methods can pinpoint where problems occur in complex systems: The study demonstrates that explainability techniques (Integrated Gradients and SHAP) can identify which specific subsystem in a chemical process is responsible for a fault, moving beyond simply knowing that a fault exists to understanding where it originated.

Different explanation methods can yield different insights: While both XAI methods generally agreed on important features, SHAP was found to be "more informative and closer to the root cause of the fault" in some cases, suggesting that using multiple explanation approaches may provide a more complete picture of AI decision-making.

Explainability techniques are transferable across similar applications: Because the XAI methods tested are "model-agnostic," the approach can be applied to other chemical processes and similar industrial systems beyond the specific benchmark tested, increasing the practical value of this research for responsible AI deployment.

Explainability complements accuracy in critical systems: The research shows that highly accurate AI classifiers (LSTM models) become more trustworthy and actionable when paired with explainability methods, which is particularly important in safety-critical domains like chemical manufacturing where understanding why a fault was detected matters as much as detecting it correctly.