Semantic match: Debugging feature attribution methods in XAI for healthcare

Feature attribution methods may be unreliable for image-based AI in healthcare, but the paper argues this limitation shouldn't discredit these explanation techniques across all data types—the problem is specific to how images are processed rather than a fundamental flaw in explainability methods themselves.

Tabular data like Electronic Health Records (EHRs) can provide meaningful explanations because individual features (e.g., blood pressure, cholesterol levels) have clear, human-understandable meanings, unlike individual pixels in medical images, making feature attribution methods more trustworthy for this data type.

"Semantic match" is the key criterion for reliable AI explanations—the paper introduces this concept to describe when AI explanations actually align with how humans understand medical information, and argues this alignment is achievable for structured health data but not necessarily for image data.

The distinction between low-level and high-level features matters for XAI reliability: low-level features (like individual pixels) lack clear meaning to doctors, while high-level features (like specific health measurements) do, which determines whether feature attribution explanations will be useful in clinical practice.

A testing procedure is needed before deploying explainable AI in healthcare: the authors sketch a method to verify whether semantic match has been achieved before relying on feature attribution explanations to support clinical decision-making.