Understanding the Impact of Physicians' Legal Considerations on XAI Systems

Physicians view themselves as the final safety buffer between AI systems and patient harm, feeling ethically, professionally, and legally responsible for outcomes—meaning they won't simply defer to AI recommendations, and XAI systems must support their decision-making rather than replace it.

Doctors anticipate AI errors but are uncertain about their legal liability, expressing unknowns about how their existing legal risk mitigation strategies will apply in cases involving AI-assisted decisions, suggesting current XAI designs don't adequately address these concerns.

Explanations alone are insufficient—physicians need XAI systems to provide AI recommendations alongside contextual information that specifically helps them understand and manage their legal risk, not just understand how the AI works.

Documentation and audit trails have legal significance that goes beyond technical transparency, meaning responsible AI design must consider how medical records, system logs, and audit requests could be used as evidence in legal cases, not just focus on explaining model outputs.

XAI system design should explicitly account for physicians' legal concerns from the start, rather than treating explainability and legal accountability as separate issues—the two must be integrated to create systems that physicians can confidently and responsibly use in clinical practice.