ExplainabilityTransparencyAccountabilitycs.CV
A novel approach to generate datasets with XAI ground truth to evaluate image models
Miquel Miró-Nicolau, Antoni Jaume-i-Capó, Gabriel Moyà-Alcover
February 11, 2023
arXiv.org
5 citations
1 influential
Key Takeaways
-
The XAI field lacks reliable verification methods: The paper identifies a critical gap in explainable AI research—there is currently no standard way to verify whether explanation techniques actually work correctly because we lack "ground truth" (known correct answers) to compare against.
-
A new method for creating trustworthy test datasets has been developed: The researchers propose a novel approach to generate datasets with verified ground truth, which would allow XAI researchers to properly validate their explanation methods rather than relying on unverified claims.
-
The proposed method shows promising validation results: The authors tested their ground truth dataset against real model explanations and reported "excellent results," suggesting this approach could become a reliable tool for evaluating whether AI explanation techniques are actually accurate.
-
Better verification of AI explanations supports responsible AI deployment: By establishing methods to verify that XAI techniques work correctly, this research helps ensure that when organizations use AI explanations to justify decisions (in hiring, lending, healthcare, etc.), those explanations are genuinely trustworthy rather than potentially misleading.
Abstract
With the increased usage of artificial intelligence (AI), it is imperative to understand how these models work internally. These needs have led to the development of a new field called eXplainable artificial intelligence (XAI). This field consists of on a set of techniques that allows us to theoretically determine the cause of the AI decisions. One main issue of XAI is how to verify the works on this field, taking into consideration the lack of ground truth (GT). In this study, we propose a new method to generate datasets with GT. We conducted a set of experiments that compared our GT with real model explanations and obtained excellent results confirming that our proposed method is correct.