Add MobiGyaan as a preferred source on GoogleOpenAI has announced a new set of child safety measures aimed at preventing the misuse of AI systems for child sexual exploitation and abuse. The initiatives include a CSAM detection classifier available through the OpenAI API, new warnings in ChatGPT to interrupt harmful behaviour, safety guidance for developers, and AI-powered tools and training for law enforcement investigators.
The measures build on OpenAI’s Child Safety Blueprint and its existing efforts to detect, block and report child sexual abuse material (CSAM). The company is also working with child protection organisations, technology providers and law enforcement agencies to strengthen responses to online child exploitation.

OpenAI Introduces CSAM Detection Classifier
One of the key announcements is a new image classifier designed to identify potential CSAM by examining visual indicators of abuse. The tool assigns scores to images to help human reviewers prioritise potentially abusive material, including AI-generated content. It can also help identify previously unseen images.
OpenAI says the classifier is the first broadly available CSAM detection tool with a specific signal indicating whether material may be AI-generated. It was trained, evaluated and developed for real-world deployment with assistance from the National Center for Missing and Exploited Children (NCMEC), which operates the CyberTipline, a centralised reporting platform for online child exploitation.
The classifier is available at no cost through the OpenAI API to vetted organisations of different sizes. OpenAI is also working with the Tech Coalition and ROOST to share access to the tool. Additionally, it supports Thorn’s Safer CSAM detection system and participates in the Tech Coalition’s Lantern programme, which enables the sharing of signals related to online child sexual exploitation and abuse.
CSAM Classifier Evaluation Results
OpenAI tested the classifier internally and with early-access partners, using their feedback to identify weaknesses and improve its performance.
In an evaluation involving user-uploaded images, OpenAI reported that the new classifier:
- Flagged an estimated 48% fewer images for review while identifying roughly the same number of abusive images as its existing tool.
- Produced flags that were about twice as likely to contain abusive material.
Early-access partners also reported results from their evaluations:
- fal: Reviewers confirmed 72% of high-confidence flags as novel CSAM in tests involving human-reviewed production images.
- invideo AI: One in four flagged cases resulted in a ban.
- SafetyKit: Detected 90% of cases involving drawings or animations that sexualise children.
OpenAI has also published a CSAM classifier model card and an intake form for organisations interested in accessing the tool. The company plans to continue improving the classifier based on feedback from participating organisations.
ChatGPT To Display Warnings Against Harmful Behaviour
OpenAI is introducing new warnings in ChatGPT to interrupt attempts to find abusive material. These warnings will explain the harm caused by sexualising children, including through AI-generated content, and direct users towards help.
When an attempt to find abusive material is flagged, ChatGPT will display a warning explaining the consequences of such behaviour. The company will also remove the flagged conversation and pause the user’s access to ChatGPT for 24 hours, while keeping specialist support available.
Repeated violations may result in longer restrictions, account closure or reports to authorities.
OpenAI said the approach was developed with input from Protect Children, the Lucy Faithfull Foundation, Dr. Caoilte Ó Ciardha, a researcher and Honorary Reader in Forensic Psychology at the University of Kent, and Joel Scanlan of the University of Tasmania, who independently evaluated a trial of the deterrent messages.
OpenAI Publishes CSAM Safety Guidance For Developers
OpenAI has also published new CSAM guidance to help developers prevent abuse, detect harmful content and respond to suspected cases. The guidance was developed with input from the Tech Coalition, Thorn, NCMEC and the Internet Watch Foundation.
It covers several areas, including:
- Establishing clear content policies and user reporting mechanisms.
- Using detection tools to review and report suspected abuse.
- Using safety identifiers to connect misuse to individual users.
- Providing appropriate support for teams reviewing sensitive material.
Developers can use the guidance alongside OpenAI’s image classifier and Moderation API, ROOST’s self-hosted review console Coop, and Thorn’s Safer CSAM detection system.
OpenAI has also worked with Anthropic and Google to jointly commission additional guidance on responding to disclosures of child sexual exploitation and abuse in AI systems. Published by the Tech Coalition, it provides recommendations for developers handling such disclosures.
The company said it has seen an increase in developers accessing child safety tools and registering with NCMEC to report CSAM.
AI Tools & Training For Law Enforcement Investigators
Another part of OpenAI’s initiative focuses on supporting specialist child-safety law enforcement units with AI tools and training.
As discussed publicly with Thorn, OpenAI said AI can help investigators organise and prioritise information while reducing unnecessary exposure to abusive material.
The support includes dedicated plugins, investigative tools and complex data analysis capabilities, with access managed by programme administrators.
The first investigative tool is designed to help officers:
- Review lengthy chat logs and legal process returns.
- Identify potential connections between information.
- Create timelines with findings linked to their original source material.
Initial training will include an overview and demonstration of the tools, along with a practical field guide for investigators. Access will depend on agency authorisation and programme requirements.
OpenAI also plans to propose additional training at regional, national and international conferences.

OpenAI’s Continued Child Safety Efforts
OpenAI said it will continue improving its safeguards and working with child protection organisations, developers, researchers and law enforcement agencies to prevent child sexual exploitation and strengthen responses when abuse occurs.
The latest measures bring together image detection, user-facing interventions, developer guidance and investigative support as part of the company’s broader approach to child safety.