Latest
AI Affairs, home

Tuesday 17 November 2026

Technology

OpenAI releases image classifier and adds ChatGPT warnings for child safety

In early partner testing, OpenAI says its classifier sent 48% fewer images for review while finding the same number of abusive images.

A hand holds a smartphone showing ChatGPT in front of a laptop.
Photo: Jernej Furman from Slovenia, CC BY 2.0, via Wikimedia Commons (cropped)

OpenAI announced an image classifier, ChatGPT warnings, developer guidance and investigative tools on 16 November 2026 as part of an effort to combat child sexual abuse material, or CSAM, TechCrunch reported. The classifier is being offered at no cost through an API to select companies, giving them a way to screen images, including AI-generated ones. In ChatGPT, attempts to request material that sexualises children will trigger a warning and a 24-hour account suspension.

Key points

  • OpenAI says its classifier flagged 48% fewer images in early partner testing while finding the same number of abusive images.
  • ChatGPT will warn users who attempt prompts that sexualise children and suspend their accounts for 24 hours.
  • The warnings are rolling out to personal accounts; enterprise and educational accounts are outside that rollout.

OpenAI’s classifier screens unseen images

The new classifier examines visual signs of abuse in images it has not previously encountered, then assigns a score intended to help reviewers assess them. A newly generated image has no earlier match among the digital signatures of previously identified material, but its visual content can still be examined. OpenAI says it developed the classifier with the National Center for Missing and Exploited Children, or NCMEC, the US clearinghouse that routes reports of child sexual abuse material to law enforcement.

In early testing with partners, OpenAI says the classifier flagged 48% fewer images while identifying the same number of abusive images. It also says images the classifier flagged were twice as likely to be abusive. Fewer images sent to reviewers, with the same number of abusive images found in that testing, would reduce the work of examining false alarms.

Sharing a picture could be subject to a check of what the image depicts even if that particular picture had never been seen before. A search that depended only on matching it to an already identified image would not serve the same purpose.

OpenAI is working with the Tech Coalition and ROOST to share access to its classifier, according to TechCrunch. Crypto Briefing reported that OpenAI has also integrated external image classifiers into its systems, including a CSAM classifier from Thorn, a nonprofit that develops child-safety technology. That account describes the Thorn classifier handling known images through hash matching as well as flagging new material. Hash matching compares an image’s digital signature with signatures of previously identified material, whereas visual screening can flag material that has not been catalogued.

ChatGPT adds a 24-hour suspension

ChatGPT already blocks CSAM and requests related to it. The new warning adds an intervention aimed at the attempt itself: when a user tries a prompt that sexualises children, ChatGPT will display an alert, point towards specialised support intended to stop harmful behaviour and suspend the account for 24 hours. OpenAI says repeated offences could result in permanent account removal. The warning therefore reaches beyond refusing a particular request to imposing a temporary consequence on the account.

OpenAI developed the alerts and guidance with Protect Children, the Lucy Faithfull Foundation and Joel Scanlan at the University of Tasmania, who conducted a separate assessment of a test of deterrent wording. Crypto Briefing reported that the interventions began a gradual worldwide rollout to personal accounts in mid-November 2026. They do not yet apply to enterprise or educational accounts, or to certain desktop interfaces.

The account-level approach follows an earlier policy effort. OpenAI released a Child Safety Blueprint on 8 April 2026 with NCMEC and other organisations, arguing that legal frameworks needed updating to deal with AI-generated CSAM. The company has also faced lawsuits alleging that its technology harmed users, particularly children.

API guidance and law enforcement tools

For developers building on its tools, OpenAI is publishing guidance on preventing and detecting CSAM and helping users report it. Crypto Briefing reported that OpenAI published CSAM guidance for API developers on 15 November 2026, covering perceptual hash matching alongside classifiers for new content. Perceptual hashing allows a stored image to be recognised after changes such as resizing, cropping or light alteration, whereas a classifier can examine an image for abusive content even when no earlier version has been recorded.

OpenAI says early testing of its guidance helped some developers adopt child-safety tools and register to report CSAM to NCMEC. For companies building products on the API, the guidance sets out two complementary checks: matching material already identified and assessing unfamiliar images.

OpenAI is also providing law enforcement personnel working on child safety with dedicated plugins, investigative tools and complex data analysis, alongside training and a field guide on using AI. Access to those tools is managed by programme administrators.

Sources

Topics: Foundation models, Safety

What should we cover? Tell us

The weekly briefing

The AI Affairs weekly briefing: the week's 10 most important AI stories in business, government and markets, every Monday. We'll email you a link to confirm. Every email has a one-click unsubscribe. Privacy policy