Explaining Low Perception Model Competency with High-Competency Counterfactuals

AI systems can now explain why they're uncertain, not just how confident they are. The paper develops methods to generate "counterfactual images"—modified versions of inputs that show what would need to change for a model to make a confident prediction—helping users understand the specific reasons behind a model's low confidence rather than just receiving an uncertainty score.

Six identifiable causes of AI model failure can be visualized and explained. The research identifies and tests against six known reasons why image classifiers lose confidence (such as image quality issues, occlusion, or unusual angles), demonstrating that counterfactual explanations can pinpoint which specific problem is causing the model's uncertainty in particular cases.

Combining visual explanations with AI language models improves human understanding. When counterfactual images are paired with large language models to generate text explanations, the resulting descriptions of why a model is uncertain become significantly more accurate and useful—suggesting that multimodal explanations (combining images and text) are more effective than either alone.

Three technical approaches show the most promise for practical deployment. Among five methods tested, Autoencoder Reconstruction, Latent Gradient Descent, and Latent Nearest Neighbors performed best, providing a foundation for real-world applications where stakeholders need transparent explanations of AI system limitations.

This work addresses a critical gap in AI transparency. Most explainability research focuses on explaining correct predictions, but this paper tackles the equally important problem of explaining when and why AI systems fail—essential information for responsible deployment in high-stakes domains.