Explainability for Embedding AI: Aspirations and Actuality

Developers need better tools to understand AI systems they build: The paper's surveys with software developers reveal a clear demand for explainability tools, indicating that developers recognize AI's complexity as a significant challenge when embedding AI into software systems.

Current explainability tools are inadequate: Despite the aspirations for Explainable AI (XAI), existing tools "still fall short" of what developers actually need, creating a gap between what's available and what's required for responsible AI development.

Explainability supports critical development tasks like debugging: The research identifies that better explanations of how AI systems work can help developers debug problems more effectively, which is essential for maintaining software quality and safety.

There is an "unmet need" for developer support mechanisms: The paper concludes that the software development community lacks "adequate support mechanisms" to help developers manage AI complexity, suggesting that current XAI solutions are not sufficiently practical or accessible for real-world use.

AI explainability is now a core software development skill: The paper frames explainability as an "essential skill" for developers, indicating that as AI becomes embedded in everyday software, understanding and explaining AI systems should be treated as a fundamental competency rather than a specialized concern.