SafetyAccountabilityTransparencycs.CY
AI Failures: A Review of Underlying Issues
Debarag Narayan Banerjee, Sasanka Sekhar Chanda
July 18, 2020
arXiv.org
11 citations
Key Takeaways
-
Design flaws are a primary cause of AI failures: The paper identifies that AI systems fail due to both "omission errors" (missing important features or safeguards) and "commission errors" (flawed implementation) in how systems are designed, suggesting that careful planning and quality control during development are critical to preventing failures.
-
AI systems struggle with moral and ethical judgments: A key finding is that AI systems are particularly prone to failure when asked to make decisions requiring moral judgment—a capability the paper notes AI fundamentally does not possess—highlighting the importance of human oversight in ethically sensitive applications.
-
Hardware limitations can undermine software quality: Even well-designed AI software can fail if the underlying hardware cannot perform reliably across different environments and conditions, indicating that responsible AI deployment requires attention to the full technical infrastructure, not just algorithms.
-
Input interpretation matters as much as the algorithm: The paper emphasizes that AI failures often stem from "failure to develop an appropriate interpretation of input information," suggesting that how systems understand and process real-world data is as important as the AI model itself.
-
Trade-offs exist in safety measures: The authors note that different approaches to mitigating AI failures involve trade-offs, implying that organizations cannot simply implement every possible safeguard and must make strategic choices about which risks to prioritize.