ExplainabilityBiasAccountabilitycs.CV
Automated Processing of eXplainable Artificial Intelligence Outputs in Deep Learning Models for Fault Diagnostics of Large Infrastructures
Giovanni Floreale, Piero Baraldi, Enrico Zio, Olga Fink
March 19, 2025
Engineering applications of artificial intelligence
6 citations
Key Takeaways
-
AI systems can develop hidden biases and rely on shortcuts that don't reflect real-world problems: The paper demonstrates that deep learning models used to inspect critical infrastructure (like power grid insulators) can make correct predictions for the wrong reasons, a risk that applies broadly to AI systems making important decisions.
-
Explainability tools alone aren't enough without automation: While XAI techniques like GradCAM can reveal how AI models make decisions, manually reviewing these explanations is time-consuming and error-prone for human experts—highlighting the need for automated quality checks on AI explanations themselves.
-
Automated explanation analysis can significantly reduce human workload: The proposed framework flagged only 15% of images for manual review while improving accuracy by 8%, meaning maintenance operators can focus their expertise on genuinely problematic cases rather than reviewing all decisions.
-
This approach is practical for high-stakes infrastructure monitoring: By combining explainability with anomaly detection, the framework enables safer deployment of AI in critical applications like power grid maintenance, where failures have real consequences for public services.
-
Responsible AI requires multiple layers of verification: The paper shows that effective AI governance in infrastructure requires not just accurate predictions, but also automated systems to verify that those predictions are based on sound reasoning—a principle applicable across sectors relying on AI for important decisions.