Skip to main content
Mallory
← Back to News
ai-platform-security

OpenAI Offers Free CSAM Detection API and Strengthens ChatGPT Child-Safety Controls

Emerging Technology Risk · 2 sources · Updated 4h ago · First seen Nov 17, 2026

OpenAI Offers Free CSAM Detection API and Strengthens ChatGPT Child-Safety Controls

OpenAI introduced an image classifier designed to identify potential child sexual abuse material (CSAM), including AI-generated images, and is offering it free through an API to vetted organizations. Early internal testing reduced the number of images requiring human review by 48% while identifying roughly the same amount of abusive material as its existing system. The classifier supports rather than replaces human reviewers; partner evaluations used different measures, and the preliminary results do not establish industry-wide effectiveness.

OpenAI is also introducing stronger ChatGPT interventions for users seeking material that sexualizes children, including conversation removal, 24-hour suspensions, and referrals to specialist support. The broader initiative includes developer safety guidance and AI-powered tools for authorized child-safety investigators. Organizations considering the classifier should evaluate its performance against their own content and review workflows rather than assume the reported reduction in review volume will generalize to their environments.

New surface, new adversariesTrack how attackers are adapting to this technology.
Track this risk
Event timeline

How this story unfolded

Events from the most recent confirmed update back to the earliest known activity.

6 events
  1. Reported Nov 17, 202610h ago

    OpenAI equips authorized child-safety investigators with AI analysis tools

    OpenAI is equipping authorized investigators with AI-powered tools to analyze lengthy chat logs and legal records. The tools can identify connections and build timelines linked to supporting evidence.

    OpenAI Launches AI Tool to Detect Child Abuse Images
  2. Reported Nov 17, 202610h ago

    OpenAI publishes developer guidance on preventing and responding to CSAM

    OpenAI published guidance covering CSAM prevention, detection, user reporting, and escalation procedures. NCMEC, Thorn, the Tech Coalition, and the Internet Watch Foundation contributed input.

    OpenAI Launches AI Tool to Detect Child Abuse Images
  3. Reported Nov 17, 202610h ago

    OpenAI introduces stronger ChatGPT interventions for child-sexualization requests

    OpenAI is introducing warnings, flagged-conversation removal, and 24-hour access suspensions for users seeking material that sexualizes children, alongside referrals to specialist support. Repeated violations may lead to longer suspensions, account closure, or reports to authorities.

    OpenAI Launches AI Tool to Detect Child Abuse Images
  4. Reported Nov 17, 202610h ago

    OpenAI introduces free CSAM detection API for vetted organizations

    OpenAI introduced an image classifier to identify potential CSAM, including AI-generated images, and offered it free through an API to vetted organizations with NCMEC's support. The tool supports rather than replaces human reviewers; OpenAI is also working with the Tech Coalition and ROOST to expand access.

    OpenAI Launches AI Tool to Detect Child Abuse Images
  5. Reported Nov 17, 202610h ago

    Partners evaluate OpenAI's child-safety image classifier

    In separate evaluations, fal reviewers confirmed 72% of high-confidence flags as previously unidentified CSAM, invideo AI banned accounts in one-quarter of flagged cases, and SafetyKit detected 90% of tested cases involving drawings or animations sexualizing children. These evaluations measured different outcomes and are not directly comparable.

    OpenAI Launches AI Tool to Detect Child Abuse Images
  6. Reported Nov 17, 202610h ago

    OpenAI tests CSAM classifier against its existing detection system

    OpenAI reported that early internal testing reduced images requiring human review by 48% while identifying roughly the same amount of child sexual abuse material (CSAM) as its existing system.

    OpenAI Launches AI Tool to Detect Child Abuse Images
Linked entities

Related entities

Vulnerabilities, threat actors, malware, products, organizations, and breaches Mallory has linked to this story.

Open in app →

Organizations

1 linked
Openai
Do you run this?Follow how adversaries are adapting to this technology, and where it touches your stack today.
Track This Risk
Mallory platform

See the full picture, correlated to your attack surface.

This page covers what’s public. Mallory adds the parts that aren’t: which of your assets are affected, which threat actors are using it right now, and what to deploy next.

Exposure mappingMap indicators from this story to your assets and identify affected systems in minutes.
Threat actor evidenceEvery observed campaign, victim, and pivot linked to actors named in this story.
Associated malwareMalware, exploits, and IOCs connected to the activity described here.
Detection signaturesYARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Scheduled alertsGet matching new stories delivered to your team as they break, not the next morning.
AI threadsAsk questions about this story and take action on the answers.