ExplainabilityTrustworthinessAccountabilitycs.HC
Trust Explanations to Do What They Say
Neil Natarajan, Reuben Binns, Jun Zhao, Nigel Shadbolt
February 14, 2023
arXiv.org
2 citations
Key Takeaways
-
Explanations themselves need accountability: Just as AI algorithms require contracts specifying when they can be trusted, explanation systems (xAI algorithms) also need explicit contracts defining their limitations. Since explanations come from algorithms, they inherit the same trustworthiness problems they're meant to solve.
-
"Trust but verify" isn't enough: The paper argues that both over-trusting and under-trusting AI systems are problematic. Simply providing explanations doesn't automatically calibrate trust appropriately—developers must actively specify which use cases explanations are reliable for and which they are not.
-
Explanations can create false confidence: Automated explanations are often presented as a solution to build trust in AI, but without clear boundaries on their validity, they may actually increase inappropriate trust. Users might accept an explanation at face value without understanding when that explanation is unreliable.
-
Calibrated trust requires transparency about limitations: Rather than assuming explanations build warranted trust, developers should provide explicit "contracts" documenting the conditions under which their explanations can and cannot be trusted, similar to what's needed for the underlying algorithms themselves.
-
Responsible AI deployment demands specificity: Vague assurances about trustworthiness are insufficient. Developers must specify concrete use cases and contexts where both algorithms and their explanations are reliable, enabling users to make informed decisions about when to rely on AI recommendations.