"Is This Really a Human Peer Supporter?": Misalignments Between Peer Supporters and Experts in LLM-Supported Interactions

AI tools can help train peer supporters, but they reveal significant gaps in current training practices. The study found that while peer supporters engaged meaningfully with an LLM-supported training system, mental health experts consistently identified critical issues in their responses—such as missing distress cues and giving advice too quickly—suggesting that existing peer support training may not adequately prepare people for emotionally complex situations.

There's a dangerous misalignment between what peer supporters think they're doing well and what experts recognize as problematic. Both peer supporters and experts saw potential in AI-supported training, but experts flagged safety concerns that peer supporters themselves didn't recognize, raising questions about whether peer supporters can reliably self-assess their own competence in high-stakes mental health interactions.

Standardized, evidence-based training is urgently needed as peer support expands globally. The paper emphasizes that as AI makes peer support more accessible worldwide, the variability in training quality and effectiveness becomes a serious concern—without standardized, psychologically grounded training protocols, scaling peer support could inadvertently put vulnerable people at risk.

Real-time AI feedback during peer support interactions shows promise but requires careful implementation. The system's emotion visualizations and LLM-generated suggestions helped improve interaction quality, but the study suggests these tools work best when paired with rigorous training standards rather than as standalone solutions.