VERA-MH Concept Paper

Clinical experts are designing safety standards for mental health AI: Practicing clinicians and academic experts developed the evaluation rubric based on established best practices for suicide risk management, ensuring that safety criteria reflect real clinical knowledge rather than technical assumptions alone.

Automated testing uses AI to simulate realistic patient interactions: The system employs two AI agents—one that role-plays patients with different risk profiles and another that scores the chatbot's responses—allowing for scalable evaluation of how mental health chatbots handle sensitive conversations about suicide risk.

Current evaluation is preliminary and undergoing validation: The researchers have only conducted initial testing on GPT-5, Claude Opus, and Claude Sonnet, and are actively working with mental health clinicians to validate whether the simulated patient interactions are realistic and whether the scoring accurately reflects clinical quality—meaning conclusions about these systems' safety are not yet final.

The field is still developing standards for responsible mental health AI: By openly seeking community feedback on both technical and clinical aspects, the researchers acknowledge that there is no established consensus yet on how to evaluate AI chatbots in mental health contexts, highlighting an important gap in responsible AI deployment.

Suicide risk assessment is the initial focus, not comprehensive mental health care: The evaluation specifically targets suicide risk management rather than broader mental health support, meaning this framework addresses one critical safety dimension but doesn't yet evaluate other important aspects of responsible mental health AI.