On the Explanation of Machine Learning Predictions in Clinical Gait Analysis

Black-box AI models in healthcare need explanation tools: The paper demonstrates that while machine learning performs well at classifying gait disorders, doctors and patients cannot understand why the model made its decisions—a critical gap that Explainable AI (XAI) methods like Layer-wise Relevance Propagation can help address.

XAI explanations can align with clinical expertise: When the researchers had a clinical expert evaluate the model's explanations for gait classification, the AI's reasoning matched established biomechanical knowledge about how different disorders affect walking patterns, suggesting the model learned clinically meaningful features rather than spurious correlations.

Multiple evaluation methods strengthen trust in AI explanations: The study used both statistical analysis and expert review to validate explanations, rather than relying on a single metric—this dual approach provides stronger evidence that an AI system's reasoning is sound and trustworthy for clinical use.

Technical choices affect both accuracy and explainability: The paper found that decisions about data normalization and which signal measurements to use impacted not only how well the model classified gait disorders, but also the quality and interpretability of its explanations, highlighting that responsible AI requires attention to implementation details.

Explainability enables clinical validation: By making the model's decision-making process transparent, clinicians could verify whether the AI was using appropriate biomechanical indicators to distinguish between disorders, rather than having to trust a black-box system—essential for safe deployment in healthcare.