A New Perspective on Evaluation Methods for Explainable Artificial Intelligence (XAI)

The explainability-performance trade-off is more complex than commonly assumed. Rather than accepting that making AI systems more explainable automatically reduces their accuracy, the paper argues this relationship requires nuanced evaluation depending on specific circumstances, challenging a widespread assumption in AI development.

Explainability should be treated as a quality requirement alongside performance. The paper positions explainability as a non-functional requirement that impacts overall system quality, meaning organizations need formal processes to evaluate and balance it against other priorities like accuracy, similar to how they manage other system requirements.

Context matters when deciding between explainability and performance. The paper identifies three key factors that should guide trade-off decisions: available resources, characteristics of the specific domain or application, and the level of risk involved—suggesting that a healthcare AI system might prioritize explainability differently than a recommendation system.

Better evaluation methods are needed for explainability in AI systems. The paper calls for improved frameworks to assess explainability, implying that current methods for measuring whether an AI system is truly understandable to users may be inadequate, which has implications for regulatory compliance and user trust.