Investigating Explainability of Generative AI for Code through Scenario-based Design

Explainability for generative AI requires different approaches than existing methods: The paper finds that explainability frameworks developed for discriminative AI models (which make decisions) don't directly transfer to generative AI models (which create new artifacts like code). This suggests the responsible AI field needs tailored explainability solutions for different types of AI systems.

Software engineers have specific, practical explainability needs: Through workshops with 43 software engineers, the research identified that users want to understand why AI-generated code was produced the way it was, particularly for critical tasks like code translation and auto-completion. This demonstrates that explainability requirements vary significantly by use case and user expertise.

Human-centered design is essential for developing trustworthy AI tools: The paper demonstrates that involving actual users (software engineers) in the design process through scenario-based workshops revealed explainability needs that might not have been anticipated by AI developers alone, suggesting responsible AI development requires direct stakeholder engagement.

Four types of XAI features were proposed for generative code models: The researchers identified and proposed design categories for explainability features tailored to generative AI for code, providing a foundation for developers to build more transparent AI coding tools that users can better understand and verify.