Ground Truth Evaluation of Neural Network Explanations with CLEVR-XAI

XAI methods have lacked objective evaluation standards: The paper identifies a critical gap in the field—explanation methods (heatmaps) were previously validated mainly through subjective human assessment or indirect proxy measures, making it unclear which methods actually work best or whether explanations could be trusted.

A new benchmark reveals surprising findings about popular XAI methods: Using the CLEVR-XAI framework with ground truth data, the researchers compared ten explanation methods and found results that "sometimes contradict with conclusions from previous comparative studies," suggesting prior evaluations may have been misleading.

Ground truth evaluation is now possible for visual explanations: The paper demonstrates that objective, controlled testing of explanation methods is feasible by using a specially designed visual question-answering task, providing a more reliable way to assess whether AI systems are actually looking at the right parts of images when making decisions.

Better evaluation frameworks support accountability and trust: By establishing standardized, objective measures for explanation quality, this work addresses a key requirement for responsible AI deployment—enabling stakeholders to verify that explanations are genuinely trustworthy rather than simply appearing plausible.