Explainable AI for Jet Tagging: A Comparative Study of GNNExplainer, GNNShap, and GradCAM for Jet Tagging in the Lund Jet Plane

Multiple explanation methods reveal consistent but imperfect alignment with physics: The study compared three different explainability approaches (GNNExplainer, GNNShap, and GradCAM) and found they generally correlate with established physics observables like N-subjettiness ratios and energy correlation functions, suggesting the AI models have learned genuine physical principles rather than spurious patterns.

AI explanations vary significantly depending on physical conditions: The quality and focus of explanations shifted noticeably between different momentum regimes (500-700 GeV vs. 800-1000 GeV), indicating that the same AI model may rely on different physical features in different situations—important context for trusting its predictions across varied real-world conditions.

Physics-informed evaluation goes beyond standard metrics: Rather than relying solely on technical fidelity measures, the researchers developed a framework that validates explanations against known physics principles and Monte Carlo simulations, demonstrating that responsible AI evaluation in specialized domains requires domain expertise, not just algorithmic benchmarks.

Explainability methods don't fully capture model reasoning: While the explanations correlated with known physics features, the correlation was imperfect, suggesting these methods reveal some but not all of what the neural network learned—a crucial caveat for anyone relying on explanations to understand or validate AI system decisions.