Developing Guidelines for Functionally-Grounded Evaluation of Explainable Artificial Intelligence using Tabular Data

XAI techniques designed for images and text may not work well for tabular data: The paper identifies a significant gap where explainability methods are often developed for visual and textual information, but their effectiveness when applied to spreadsheet-like data (the most common format in business and healthcare) remains unclear and largely unevaluated.

How we test XAI methods matters as much as the methods themselves: The research finds that the evaluation procedures used to assess explainability techniques can actually influence the results, meaning poorly designed tests could give false confidence in explanations that aren't actually reliable—a critical concern for real-world deployment.

Current evaluation standards lack clear guidance: The paper identifies 20 different evaluation criteria for XAI but finds no consensus on when and how each should be used, creating confusion about which explanations are genuinely trustworthy versus which just appear to be through flawed testing.

Transparency claims need functional grounding: Rather than accepting explanations at face value, the guidelines emphasize that XAI evaluation should be "functionally-grounded"—meaning explanations must be tested to verify they actually help users make better decisions or understand model behavior in practice, not just theoretically.

Significant research gaps remain before XAI can be reliably deployed: The study highlights that without standardized evaluation protocols for tabular data, organizations using AI in real-world settings (banking, healthcare, insurance) may be deploying explainability tools that provide false confidence rather than genuine transparency.