Skip to content

OpenAI gives platforms a free CSAM detection classifier

Approved organisations can now call a new OpenAI image classifier through its API at no cost, built with the group behind the US CyberTipline to flag child sexual abuse material for review.

Source: OpenAI

OpenAI released an image classifier built to help platforms identify child sexual abuse material, including AI-generated CSAM, and is making it available through its API at no cost to approved organisations. The company built it with the National Center for Missing and Exploited Children, and is distributing access through the Tech Coalition and ROOST rather than opening it to every developer.

Getting access means applying, passing identity verification through Persona, and agreeing to the programme’s terms, according to OpenAI’s guidance page. In OpenAI’s own testing, reported by TechCrunch, the classifier flagged 48% fewer images than a comparison method while catching the same number of genuinely abusive ones, and flagged images were twice as likely to actually be CSAM. Separately, ChatGPT now warns and temporarily suspends accounts, for 24 hours, that try to access this kind of material, with repeat attempts leading to permanent removal.

Why it matters: any platform that accepts user-uploaded images carries this moderation burden, not just chat apps, and building a classifier like this in-house is expensive and slow to get right. A free, API-accessible option from a major lab, vetted through NCMEC, gives smaller platforms a path to this kind of review that most could never build on their own.

The caveat: access sits behind an approval process, not a public API key, so most developers reading this cannot call it directly today, even though the underlying problem is one plenty of platforms with user uploads will eventually hit.

Share

Related reading

More from The Wire

All briefs