Skip to content
Trending storyDeveloping

OpenAI releases CSAM classifier with supporting safeguards

3 reports3 sourcesupdated 4 hours ago

What happened

AI digest

On November 17, OpenAI announced a child sexual abuse material (CSAM) image classifier that detects potential CSAM, including AI-generated material. The API is free to vetted organizations, alongside ChatGPT interventions, developer guidance and support for law enforcement investigations. TechCrunch reported the tool was built with NCMEC. On results, OpenAI said earlier that flagging is expected to drop 48% while catching about the same number of abuse images; TechCrunch later reported that in early partner tests detections held steady, flags fell 48%, and flagged images were twice as likely to contain abuse material. The Verge then reported the tool launched at a Washington, D.C. event and can also tell whether an image is AI-generated. ChatGPT's image generator will show a new warning to users attempting such content, with a 24-hour access pause on a first violation and notice that the account may be permanently closed. OpenAI also helped train law enforcement.

Written by AI from the coverage · updated 3 hours ago

Coverage

Follow the reports to see the story from different sides.

Nov 17
  1. The Verge · AI
    OpenAI wants to stop deepfakes from overwhelming child abuse tiplines

    OpenAI 在华盛顿特区发布会上推出一款分类器,称能更准确识别儿童性虐待图像并判断是否为 AI 生成,标记的图像数量减少 48%,识别出的图像中含虐待材料的比例提高一倍。该工具向通过审核的机构免费开放,OpenAI 同时为开发者提供识别指引并协助培训执法人员。ChatGPT 图像生成器对试图生成此类内容的用户将显示新警告,首次违规会触发 24 小时访问暂停并提示账号可能被永久关闭。

  2. TechCrunch · AI
    OpenAI releases a new image classifier and ChatGPT warnings to combat CSAM

    OpenAI 发布一款可检测 CSAM 图像(含 AI 生成图像)的分类器,通过 API 免费向部分公司开放,并与 NCMEC 共同开发。早期合作测试中,该分类器标记的图像减少 48%,检出的虐待图像数量不变,被标记图像为虐待图像的概率提升一倍。

  3. OpenAI NewsPick
    OpenAI releases CSAM image classifier with ChatGPT safeguards and law enforcement support

    OpenAI released a child sexual abuse material (CSAM) image classifier, along with ChatGPT interventions, developer guidance and support for law enforcement investigations. The classifier detects potential CSAM, including AI-generated material, and is offered as free API access to vetted organizations. Early evaluations estimate it flags 48% fewer images while finding roughly the same number of abuse images.

Heat over time

Heat now 26·Comparable peak 27(Nov 17 05:00)·Comparable change over 24 hours –

0102030Nov 1702:00Nov 1704:00Nov 1705:00Nov 1707:00

The trend only compares accounts observed without gaps, so its range may be smaller than the current heat. Hover or tap the chart for each hour; the left and right arrow keys step through it.