ExplainabilityTrustworthinessAccountabilitycs.LG
It is not "accuracy vs. explainability" -- we need both for trustworthy AI systems
D. Petkovic
December 16, 2022
IEEE Transactions on Technology and Society
54 citations
1 influential
Key Takeaways
-
Both accuracy and explainability are essential, not competing priorities: The paper challenges the false choice between building highly accurate AI systems and building explainable ones, arguing that trustworthy AI requires achieving both simultaneously rather than sacrificing one for the other.
-
Real-world AI failures demonstrate the cost of opacity: The paper documents concrete examples of AI system failures including accidents with self-driving cars, biased hiring and healthcare decisions, and face recognition errors affecting people of color—cases where lack of explainability prevented detection of problems until after harm occurred.
-
Regulatory and governance frameworks now demand transparency: Multiple government and regulatory initiatives are requiring AI systems to provide not just accuracy but also explainability, robustness, bias elimination, human oversight, and judicial transparency, making explainability a practical necessity rather than an optional feature.
-
The explainability challenge extends beyond deep learning: The paper pushes back against the assumption that explainability concerns apply only to complex deep learning models, suggesting the need for transparency spans broader categories of AI systems used in high-stakes domains.
-
Trust in AI adoption depends on understanding how decisions are made: Beyond technical performance metrics, people need to understand the reasoning behind AI decisions to trust and responsibly adopt these systems in critical areas like healthcare, transportation, and employment.