XAI for Coding Agent Failures: Transforming Raw Execution Traces into Actionable Insights

AI coding agents fail frequently and unpredictably: The paper demonstrates that LLM-based coding agents commonly encounter failures that are difficult for developers to understand, even when using general-purpose AI explanations like those from GPT, highlighting a significant gap in current AI transparency.

Structured explanations dramatically improve human understanding: Users were able to identify the root causes of AI failures 2.8 times faster when using the paper's systematic approach compared to raw execution traces, and proposed correct fixes with 73% higher accuracy, showing that how failures are explained matters enormously for practical debugging.

Visual and natural language explanations work better than AI ad-hoc responses: The paper's hybrid approach—combining visual execution flows, written explanations, and specific recommendations—outperformed state-of-the-art AI models at explaining failures, suggesting that domain-specific structure and visualization are more valuable than general-purpose AI explanations alone.

Non-technical users benefit equally from better explanations: The user study included both technical and non-technical participants, and the structured approach improved understanding across both groups, indicating that responsible AI design can make complex system failures accessible to diverse stakeholders, not just experts.

Systematic failure categorization is foundational for AI accountability: By developing a taxonomy of real failure types observed in coding agents, the researchers created a reusable framework for understanding and addressing AI mistakes, establishing a model for how organizations can systematically analyze and improve AI system reliability.