Redistributing Voice and Responsibility: AI in Relationship-Centred Care

AI in healthcare isn't primarily about making better decisions—it's about changing who gets to make them. The research found that in sensitive care contexts like advance care planning and peer support, AI's main effect is redistributing "voice, responsibility, and emotional labour" among patients, caregivers, and providers, rather than simply optimizing outcomes.

Patients and caregivers care more about relationships than accuracy. Across both settings studied, participants were "less concerned with technical accuracy than with relational consequences"—they worried about whether AI would properly represent their values, reduce their burden, or create confusion about who is actually responsible for decisions.

AI can subtly displace human connection in care. The paper identifies three critical ways AI reshapes care relationships: through authority (who decides), temporality (when decisions happen), and visibility (who sees what). Designers must ensure AI "scaffolds connection" rather than replacing the human relationships that are central to quality care.

We need to know who is responsible when things go wrong. The research emphasizes "responsibility traceability" as essential—in high-stakes care decisions, it must be clear who bears moral responsibility, not obscured by the presence of an AI system in the decision-making process.

AI should have "bounded agency" in care contexts. Rather than giving AI systems broad decision-making power, the authors propose design approaches that keep AI in a supporting role, ensuring it enhances rather than substitutes for the human judgment and relationships that define good care.