Interpretable and Explainable Surrogate Modeling for Simulations: A State-of-the-Art Survey and Perspectives on Explainable AI for Decision-Making

Surrogate models inherit opacity from the complex systems they approximate, making it difficult to understand how input variables actually drive outcomes—a critical problem when these models inform real-world decisions in engineering and science.

Explainable AI (XAI) methods exist but aren't well-suited to engineering challenges such as highly correlated inputs, dynamical systems, and the need for rigorous reliability requirements, meaning current XAI tools often fail when applied to practical simulation problems.

Surrogate modeling and explainability have developed separately despite being complementary, suggesting that intentionally combining these fields could help create models that are both computationally efficient and understandable to human decision-makers.

Key open challenges remain unsolved, particularly explaining how dynamical systems (systems that change over time) work and handling mixed-variable systems, indicating that responsible AI in simulation requires continued research before these tools can be reliably deployed for high-stakes decisions.

Making explainability a core feature of surrogate modeling workflows from the design stage onward—rather than adding it afterward—could help ensure that complex simulations used in engineering and scientific decision-making remain transparent and trustworthy.