OpenAI adds image detection and ChatGPT warnings against child abuse
OpenAI is offering selected organisations a tool intended to flag suspected child sexual abuse imagery, including newly created material. ChatGPT will also display deterrent warnings and direct users towards support. The announcement joins detection with account restrictions and developer guidance, adding interventions alongside existing prohibitions on abusive content rather than opening an unrestricted public upload service.
Artificial Intelligence··Night
New images can be sent for human review
OpenAI has announced an image classifier intended to detect suspected child sexual abuse material, including imagery made with artificial intelligence. It is also adding warnings in ChatGPT.[1], [2]
The tool is offered free to selected organisations through an application programming interface. It assigns images scores to help human reviewers prioritise attention. This complements digital-fingerprint matching, which recognises already identified files. Work on the classifier involves the National Center for Missing and Exploited Children, a US child-protection organisation.[1]
ChatGPT warnings direct users towards support
The new warning screens describe the harm and point people seeking prohibited content towards specialist support. Account access can be suspended for 24 hours, while accounts can be permanently removed after repeated violations. OpenAI developed the messages with child-protection organisations and researchers, including the Lucy Faithfull Foundation, which works to prevent child sexual abuse.[1]
Developer guidance covers prevention and reporting
Developer guidance addresses preventing abusive material, detecting it and reporting it to appropriate organisations. OpenAI is also providing training, a field guide and data-analysis tools for law-enforcement teams working on child safety. Programme administrators control access to those investigative tools. The classifier and account interventions accompany ChatGPT’s existing blocks on prohibited material.[1]