The UK Safer Internet Centre (UKSIC) has reported that children are using AI image generators to create indecent images of other children.
The charity said it’s reviewed a “small number of reports” but highlighted that action is required now before the problem spreads.
David Wright, the director of UKSIC, emphasizes the need for immediate action: “[We] need to see steps being taken now, before schools become overwhelmed and the problem grows.”
Children, in particular, likely don’t realize the implications of using AI in this way.
In the UK, creating, possessing, or distributing such images is illegal, regardless of whether they are real or AI-generated.
The charity warned of the potential for these images to be misused or circulated online, leading to unintended consequences, including blackmail.
This isn’t an isolated case – Spanish and US schools both experienced their own AI-generated imagery controversies lately, with illicit images of minors being exchanged.
Moreover, there’s considerable debate over who is responsible for educating children about the harms of such material. Does the duty fall on parents, schools, or governments?
Seperate research by educational technology company RM Technology involving 1,000 pupils showed that nearly one-third use AI for inappropriate online activities.
Tasha Gibson, online safety manager at RM Technology, said, “Students using AI regularly is now commonplace,” “In fact, their understanding of AI is more advanced than most teachers – creating a knowledge gap.”
Apps make it easy to create illicit imagery
These so-called “declothing” apps first appeared on social media platforms in 2019, often used on messaging services like Telegram as automated software with AI capabilities.
Initially, these apps were relatively unsophisticated, but advancements in generative AI have improved their ability to create photorealistic fake images.
The model involved in the Spanish case attracted nearly 50,000 subscribers. Typically, users can create a number of images for free before being required to pay a fee.
In many cases, the use of these apps is driven by curiosity, says UKSIC, but that certainly doesn’t constitute a legal defense.
A North Carolina man recently became probably the first individual to be sentenced for AI-related sexual crimes, receiving a 40-year prison term. He is unlikely to be the last.
