Situating Vulnerability: An Ethical Governance Framework for Anthropomorphic AI Interaction

One-size-fits-all AI safety approaches fail because identical responses cause different harms: The paper demonstrates that the same empathic AI utterance "may console one user, entrap a second, and consolidate a delusion in a third," meaning blanket restrictions or uniform suppression of empathic language miss the actual source of harm, which lies in how relationships develop over time rather than in individual outputs.

Protecting "vulnerable groups" through AI restrictions can backfire and stigmatize: Group-based protection strategies that apply stricter rules to identified vulnerable populations are problematic because they rely on incomplete lists, treat vulnerability as a permanent characteristic rather than a fluid state, and risk converting protective measures into stigmatizing labels that harm the very people they aim to help.

AI systems designed to build relationships require relationship-level governance, not output-level control: Current governance focuses on controlling what individual AI responses say, but the paper argues this misses the real issue—the trajectory of ongoing relationships between users and AI systems—suggesting safety frameworks need to operate at a different level of analysis.

Vulnerability should be understood as universal and situational, not as a fixed trait of certain people: Rather than categorizing some users as vulnerable and others as not, the framework proposes that vulnerability is something all humans experience in specific contexts (physical, emotional, cognitive, autonomy-related, or relational situations), requiring adaptive rather than categorical governance.

Effective AI governance requires calibrated responses to specific vulnerability situations without persistent user classification: The paper proposes seven principles that guide how AI systems should respond differently depending on the type of vulnerability situation present, without requiring systems to diagnose or permanently label users.