Inpainting Insights: Elevating Visual XAI with Photorealistic Perturbations

Unrealistic perturbations can undermine AI explanations: Traditional methods for explaining image-based AI decisions often create fake or distorted images when testing what happens if parts of an image are removed. These artificial images don't look like real data, which can trick the model and produce misleading explanations about why it made its decision.

Generative AI can improve explanation reliability: The researchers used generative inpainting—a technique that realistically fills in missing parts of images—to create more believable test images. This approach produced explanations that better reflect how the model actually behaves on real-world data, rather than on artificial artifacts.

Better explanations require better perturbations: The study demonstrates that the quality of AI explanations depends heavily on the quality of the test images used to probe the model. By ensuring perturbed images look photorealistic and stay within the distribution of natural images, the researchers achieved more trustworthy insights into model behavior.

Existing popular methods have a blind spot: LIME, one of the most widely used techniques for explaining AI decisions on images, was improved by addressing its perturbation strategy. This suggests that even established, trusted explanation methods may have room for improvement when it comes to generating realistic test cases.