ExplainabilityTransparencyAccountabilitycs.AI
Revisiting the Performance-Explainability Trade-Off in Explainable Artificial Intelligence (XAI)
Barnaby Crook, Maximilian Schlüter, Timo Speith
July 26, 2023
2023 IEEE 31st International Requirements Engineering Conference Workshops (REW)
40 citations
1 influential
Key Takeaways
-
The explainability-performance trade-off is not absolute: The paper challenges the common assumption that making AI systems more explainable automatically reduces their accuracy or performance. This suggests organizations shouldn't automatically accept worse results when pursuing transparency—the relationship is more complex than previously thought.
-
Context matters more than blanket rules: According to the paper, decisions about prioritizing explainability versus performance should depend on specific factors including available resources, the characteristics of the domain, and the level of risk involved. A high-stakes medical diagnosis system may warrant different trade-offs than a low-risk recommendation engine.
-
Explainability is a legitimate quality requirement: The paper establishes explainability as an important "non-functional requirement" in AI system design—meaning it's not just a nice-to-have feature but a core aspect of system quality that should be considered alongside performance from the start, similar to security or reliability.
-
Better alignment with user needs and regulation: The paper emphasizes that XAI helps align AI systems with what users actually need and what society and regulators expect, suggesting that explainability serves a broader purpose beyond just technical performance metrics.
-
Requires thoughtful compromise, not either/or thinking: Rather than choosing between explainability and performance, the paper advocates for a nuanced approach that carefully weighs these competing demands based on specific circumstances—implying that responsible AI development requires context-specific decision-making rather than universal rules.