Beyond "I Can't Help With That": How Child Safety Experts Evaluate AI Chatbot Safety

Current AI safety evaluations miss real-world harms: The paper found that existing chatbot safety tests focus on detecting adversarial attacks and surface-level problems, but fail to catch responses that practitioners identified as genuinely harmful to youth in practice. This suggests that passing current safety benchmarks doesn't guarantee a chatbot is actually safe for vulnerable young people.

Simple refusals aren't always the right answer: Rather than assuming that chatbots should simply refuse to engage with sensitive topics (e.g., "I can't help with that"), the 19 child safety experts interviewed indicated that thoughtful, supportive responses are sometimes more appropriate and helpful. The paper implies that overly rigid safety guardrails may actually harm youth by denying them meaningful support during difficult moments.

Expert practitioners should guide safety standards: The research demonstrates that social workers, therapists, and psychologists who work directly with vulnerable youth have crucial insights about what constitutes harm and help that AI developers currently lack. The paper recommends incorporating these practitioners' real-world expertise into the design and evaluation of chatbot safety systems, rather than relying solely on technical assumptions.

Context and nuance matter more than simple rules: The paper suggests that appropriate chatbot responses depend heavily on the specific situation, the youth's needs, and the severity of risk—factors that current evaluation methods often oversimplify. One-size-fits-all safety approaches may fail to account for the complexity of youth mental health and crisis situations.