Explainable and Human-Grounded AI for Decision Support Systems: The Theory of Epistemic Quasi-Partnerships

AI explanations need three components to be truly useful: The paper argues that effective AI decision support systems should provide reasons (why the AI made a decision), counterfactuals (what would need to change for a different outcome), and confidence levels (how certain the AI is). This "RCC approach" goes beyond current explanation methods like LIME and SHAP, which often fall short of what human decision-makers actually need.

Current explanation methods don't reliably improve human decision-making: The research reveals a gap between existing XAI techniques and their actual impact—generating explanations doesn't automatically make people trust AI systems more or make better decisions. This suggests that simply making AI "more explainable" isn't enough without understanding what explanations humans genuinely need.

Ethical AI requires treating humans as partners, not just users: The paper introduces "epistemic quasi-partnerships," a framework where AI systems should respect human decision-makers as knowledgeable partners rather than passive recipients of information. This means AI-DSS should be designed to support human judgment and expertise, not replace it.

Trustworthiness and accuracy are separate from explanation quality: The empirical evidence shows that people can trust an AI system without understanding it well, or understand it without trusting it. Responsible AI development must address both dimensions—explanations alone won't create appropriate trust or improve decision quality.

Responsible AI design requires rethinking human-machine relationships: Rather than focusing solely on technical explanation methods, the paper argues that ethical AI development should be grounded in a theory of how humans and machines should interact as partners in decision-making.