Peering inside the black box: Learning the relevance of many-body functions in Neural Network potentials

Neural network potentials can be made interpretable: The researchers successfully applied explainable AI (XAI) tools to "open the black box" of machine learning potentials, demonstrating that these models can be decomposed into understandable components that reveal which physical interactions the network considers most important.

Relevance contributions can be ranked by complexity: The study shows that neural network potentials can be broken down by "orders" of interaction—meaning researchers can identify which simpler versus more complex physical relationships drive the model's predictions, making the decision-making process more transparent.

The approach works across diverse systems: The method was validated on three different types of systems (methane fluid, water fluid, and a protein called NTL9), suggesting this interpretability approach has broad applicability rather than being limited to one specific domain.

Interpretability enables physical insight: By decomposing the neural network's energy predictions, researchers gained actual scientific understanding of the systems being modeled—not just accurate predictions—which is crucial for validating that AI models are learning physically meaningful relationships rather than exploiting spurious patterns.

Addresses a major criticism of AI in science: This work directly tackles one of the primary concerns about using machine learning in scientific research: that while neural networks may be accurate, their lack of interpretability makes them unsuitable for discovery and validation of scientific principles.