Notes on Applicability of Explainable AI Methods to Machine Learning Models Using Features Extracted by Persistent Homology

Persistent homology features improve interpretability: The paper finds that machine learning models using features extracted through persistent homology (a topological data analysis method) can achieve good accuracy with simpler models, which inherently makes these systems more interpretable than complex "black box" approaches.

Current interpretations have significant limitations: Despite the theoretical interpretability advantages, the paper identifies a critical flaw: the interpretations generated don't accurately reflect real-world constraints, such as the physical or chemical limitations that actually govern data generation processes—meaning explanations can be misleading even when models perform well.

Explainable AI methods can help bridge the gap: By applying explainable AI (XAI) methodologies on top of the persistent homology-machine learning pipeline, the authors demonstrate that it's possible to generate more accurate and meaningful explanations that better align with actual domain constraints.

Practical demonstration in materials science: The paper demonstrates these concepts using a real-world application—predicting gas adsorption in metal-organic frameworks—showing that the approach can produce "suggestive results" in actual scientific problems, not just theoretical scenarios.

Responsible AI requires multiple layers of scrutiny: The findings suggest that even when a machine learning approach seems interpretable by design, additional explainability techniques are necessary to ensure the explanations actually reflect reality and don't mislead users about how and why models make predictions.