Large-Scale Real-World Audit Reveals Gaps in Mental Health AI Safety Beyond Simulations
A new arXiv preprint reports an ecological audit of 20,000 real-world mental health AI conversations, comparing a purpose-built system to several leading general-purpose models. The study finds that the specialized system produced significantly fewer potentially harmful responses on sensitive topics such as suicide and eating disorders, and clinician review found no missed suicide-risk cases lacking crisis resources. The authors argue that real-world auditing provides critical safety insights not captured by standard simulation-based benchmarks.
Why it matters: The work suggests that real-world audits are essential for identifying safety issues in mental health AI that may be missed by traditional benchmark testing.
Full story at: arXiv Computers and Society ↗