VERA-MH: Validation of Ethical and Responsible AI in Mental Health

Chatbots are being used for mental health support despite not being designed for it, creating a safety gap that requires new evaluation methods. VERA-MH addresses this by providing the first clinically-validated framework specifically designed to test how well chatbots handle users expressing suicidal ideation.

The evaluation uses realistic crisis scenarios developed with clinical guidance to test chatbot responses. User personas were created to represent multiple risk factors, demographics, and disclosure patterns, ensuring the assessment reflects real-world mental health crises rather than generic conversations.

A structured clinical rubric with yes/no decision trees improves consistency and reveals failure patterns. By using a flowchart-based approach rather than open-ended evaluation, the framework makes it easier to identify exactly where and how chatbots fail to respond appropriately to users in crisis.

Four major LLM providers were evaluated using this framework, providing concrete data on how current leading chatbots perform in mental health safety scenarios—information that was previously unavailable to the public and policymakers.

This work establishes a model for responsible AI evaluation in high-stakes domains. By combining clinical expertise with AI evaluation methods, VERA-MH demonstrates how safety frameworks should be developed for AI applications in sensitive fields like healthcare and mental health support.