What Should Explanations Contain? A Human-Centered Explanation Content Model for Local, Post-Hoc Explanations

Users need multiple types of explanation content, not just one: The research identified 14 distinct categories of explanation content that industrial AI users find relevant, organized into four groups (rule-based, causal, and two types of epistemic explanations). This suggests that effective AI explanations should combine different types of information rather than relying on a single explanation approach.

Real-world user needs should drive explanation design: The study analyzed 325 meaning units from actual user studies across diverse industries (building technology, manufacturing, software development, and hospital cybersecurity), finding that different contexts and user types have varying explanation needs. This emphasizes that explanations designed in laboratories may not match what practitioners actually require.

The proposed framework provides a practical tool for building better explanations: The 14-code model achieved strong expert validation (scale-level agreement of 0.92-0.94) and high reproducibility (Krippendorff's α = 0.920), meaning organizations can use this framework to systematically plan and evaluate what their AI systems should explain to users.

Some important explanation types may be overlooked without deliberate planning: Two of the 14 explanation categories (Rule base and What-if backward) weren't found in the user study data but were added based on XAI literature and expert review, suggesting that organizations might miss valuable explanation types if they only rely on observed user requests rather than proactively considering what users might need.