XAI in the context of Predictive Process Monitoring: Too much to Reveal

XAI methods produce significantly different explanations for the same predictions: Even when using identical data, preprocessing techniques, and machine learning models, different explainability methods generate profoundly different explanations, raising questions about which explanation—if any—accurately reflects how the model actually works.

There is no clear framework for understanding what drives explanation differences: The paper identifies a critical gap in the field: researchers and practitioners lack systematic understanding of which specific settings (data choices, model types, XAI method selection) determine what kind of explanation gets produced, making it difficult to know which explanations to trust.

Transparency tools may create false confidence without solving the trust problem: While XAI methods are deployed to build stakeholder trust in predictive process monitoring systems, the inconsistency between different XAI approaches suggests that explanations alone may not genuinely increase understanding—they might instead create an illusion of transparency.

XAI method choice fundamentally shapes what users learn about model reasoning: Different explainability techniques have distinct characteristics that actively shape the resulting explanations, meaning the choice of which XAI method to use is not neutral and can lead stakeholders to different conclusions about why a prediction was made.

Organizations need systematic evaluation before deploying XAI in business processes: Before using explainability methods to justify automated predictions in real business processes, organizations should understand how their specific data, models, and XAI tools interact—otherwise they risk making decisions based on explanations that may be misleading or incomplete.