The human-AI relationship in decision-making: AI explanation to support people on justifying their decisions

AI explanations serve a critical function beyond technical transparency: The paper argues that XAI's primary value in decision-making isn't just about understanding algorithms, but about enabling people to justify their decisions to others—a social and accountability function that differs from traditional interpretability research.

Trust in AI systems requires mutual understanding: Building a healthy human-AI relationship in decision-making contexts depends on decision-makers having sufficient awareness of how AI works and why it produced specific outcomes, suggesting that one-way algorithmic transparency is insufficient.

Decision-makers need to be accountable for AI-influenced choices: When people make decisions based on or influenced by AI systems, they must be able to explain those decisions to stakeholders. This means AI explanations must be designed to support human justification, not replace human responsibility.

Current XAI research may miss the real-world need: The paper identifies a gap between existing XAI research (which focuses on algorithm interpretability and ML model transparency) and what decision-makers actually need (explanations that help them communicate and justify choices to others).

Context matters for explanation design: The paper emphasizes that different decision-making domains have different accountability requirements, suggesting that effective AI explanations must be tailored to the specific contexts where decisions are made and justified.