OpenAI introduced a new image classifier that flags potential child sexual abuse material (CSAM), including images created with artificial intelligence. The company developed the tool with the National Center for Missing and Exploited Children (NCMEC), which operates the CyberTipline for reporting online child exploitation.
OpenAI will offer the classifier free through its API to vetted organizations. It is also working with the Tech Coalition and ROOST to widen access to the tool across the technology industry.
The classifier is intended to help safety teams identify material that conventional hash-matching systems may miss. It assigns a score to an image, but OpenAI says that score alone does not confirm abuse, establish its legal status, or determine whether a report is required.
Early testing suggests the system reduces false alerts by 48 percent while maintaining detection performance. OpenAI says images it flags are twice as likely to contain abusive material, which could reduce the burden on human reviewers.
ChatGPT will now interrupt users who seek content that sexualizes children. The warning explains the potential harm, directs users to specialist support, and notes that AI-generated material can also be harmful
OpenAI will remove the flagged conversation and pause the user’s access to ChatGPT for 24 hours. Repeat or serious violations can result in permanent account closure. OpenAI said it reports detected instances of CSAM to NCMEC. The measures mark a broader attempt to prevent generative AI from becoming a channel for creating, distributing or seeking abusive material involving children.
The company also published guidance to help developers detect and respond to CSAM. It is providing law-enforcement agencies with investigative tools and training to address child exploitation involving AI.
Also Read: OpenAI, Anthropic, Google Face Lawsuit Seeking Freeze on Advanced AI