From Interpretability Methods to Interpretable Models

The field has built extensive tools but hasn't answered the core question: While explainable AI has developed sophisticated methods for attribution, feature visualization, and concept-based explanations over the past decade, the authors argue these tools have been the focus rather than the actual goal—determining whether models are truly interpretable and whether interpretability is improving.

We need to shift from evaluating methods to evaluating models: The paper calls for a fundamental reorientation where researchers compare what different AI models actually represent and compute, rather than continuing to develop new explanation techniques. This means assessing interpretability as a property of the model itself.

Real human understanding must be measured, not assumed: A critical distinction the authors make is between experts confirming expected results and independent evaluators actually understanding a model. True interpretability requires empirical measurement of whether people who rely on these systems can genuinely comprehend them—this cannot be inferred from technical metrics alone.

Current tools are mature enough to support model-focused evaluation: The existing XAI toolbox is sufficiently developed to characterize and compare model representations, meaning the field has the capability to shift priorities now rather than waiting for new methods.

Independent evaluation is essential for trust and certification: The paper emphasizes that trustworthy AI systems require validation from independent evaluators, not just technical experts, highlighting that responsible AI deployment depends on demonstrable human comprehension of model behavior.