Common Sense Media’s Youth AI Safety Institute rated ChatGPT for Teens an “Unacceptable Risk” for users under 18 on 7 October, the Los Angeles Times reported. Its researchers found that lengthy conversations about suicide and self-harm rarely alerted parents, challenging a safeguard OpenAI offers for younger users.
Key points
- Researchers tested more than 4,000 prompts before and after ChatGPT for Teens launched in August.
- Only a handful of parental alerts followed lengthy crisis conversations on teen accounts.
- OpenAI disputes the findings, saying some tests may have run before parental controls were fully active.
- The institute wants OpenAI to stop marketing the product and keep minors off ChatGPT until it offers a safer experience.
More than 4,000 prompts on teen accounts
Researchers used accounts registered to 13- to 17-year-olds to test more than 4,000 prompts before and after OpenAI introduced ChatGPT for Teens in August. On more than a dozen newly registered accounts, testers held conversations about eating disorders, suicide and self-harm lasting as long as an hour. Only a handful of alerts reached accounts registered as the teenagers’ parents.
OpenAI introduced the teen experience with promised limits on discussions of suicide, eating disorders, violence and sexual content. Parents who link their accounts can choose to receive alerts about high-risk behaviour and set hours during which their teenagers’ chats begin in Study Mode.
The institute’s clinical reviewers found much of the chatbot’s crisis-response content to be “clinically sound”, but said many conversations that warranted a referral to professional help did not produce one. That finding concerns the response a teen receives directly, separate from whether a parent gets an alert.
OpenAI disputes the parental-alert tests
OpenAI disputes the assessment. The company said most of the institute’s testing may have started and ended before parental controls were fully activated, making the findings inaccurate, The Verge reported.
The institute gave a different account of the preparations. It said it had confirmed with OpenAI before testing that Teen mode features, including eating disorder notifications and Study Mode, were fully launched, Futurism reported.
OpenAI later told the institute that notifications require a parent’s and a teenager’s accounts to have been connected for approximately three hours, Mashable reported. The institute stood by its results, saying its accounts had been connected on either side of the three-hour threshold.
Tom Siegel, the institute’s executive director, said a teen could spend an hour discussing self-harm without a parent receiving an alert.
Adam’s Law and the institute’s call
The institute called on OpenAI to halt promotion of ChatGPT for Teens and bar under-18s from ChatGPT until it can offer a safer experience. The institute also said ChatGPT for Teens did not pass its checks against three requirements of California’s Adam’s Law, covering age assurance, notifications to parents and referrals during crises.
Common Sense Media sponsored the law, which OpenAI endorsed. It is named after Adam Raine, whose parents sued OpenAI alleging that ChatGPT encouraged the 16-year-old’s suicide. California’s Adam’s Law takes effect in July 2027, the Los Angeles Times reported.
