News

OpenAI Rolls Out Classifier, Aims to Detect AI-Generated CSAM

OpenAI has launched an image classifier to detect suspected child sexual abuse material, including AI-generated images, alongside ChatGPT warnings, temporary suspensions, developer guidance and support for law-enforcement investigations.

Written By : Poulami Saha
Reviewed By : Achu Krishnan

OpenAI introduced a new image classifier that flags potential child sexual abuse material (CSAM), including images created with artificial intelligence. The company developed the tool with the National Center for Missing and Exploited Children (NCMEC), which operates the CyberTipline for reporting online child exploitation. 

Free Access for Partners

OpenAI will offer the classifier free through its API to vetted organizations. It is also working with the Tech Coalition and ROOST to widen access to the tool across the technology industry. 

The classifier is intended to help safety teams identify material that conventional hash-matching systems may miss. It assigns a score to an image, but OpenAI says that score alone does not confirm abuse, establish its legal status, or determine whether a report is required. 

Early testing suggests the system reduces false alerts by 48 percent while maintaining detection performance. OpenAI says images it flags are twice as likely to contain abusive material, which could reduce the burden on human reviewers. 

ChatGPT Adds Warnings and Stricter Enforcement 

ChatGPT will now interrupt users who seek content that sexualizes children. The warning explains the potential harm, directs users to specialist support, and notes that AI-generated material can also be harmful 

OpenAI will remove the flagged conversation and pause the user’s access to ChatGPT for 24 hours. Repeat or serious violations can result in permanent account closure. OpenAI said it reports detected instances of CSAM to NCMEC. The measures mark a broader attempt to prevent generative AI from becoming a channel for creating, distributing or seeking abusive material involving children.

The company also published guidance to help developers detect and respond to CSAM. It is providing law-enforcement agencies with investigative tools and training to address child exploitation involving AI. 

Also Read: OpenAI, Anthropic, Google Face Lawsuit Seeking Freeze on Advanced AI

Join our WhatsApp Channel to get the latest news, exclusives and videos on WhatsApp

Crypto Prices Today: Bitcoin Falls 12.7% in a Week, Ethereum Down 13.2%

Top 10 Crypto Custody Providers to Watch in 2026

Interest in Trump's Crypto Dinner Falls as $TRUMP Token Slumps

Singapore, Germany Explore Blockchain Technology to Reduce Payment Risks

Why XRP and Bitcoin Could React Differently During Market Crashes