Prominent social media platforms – Facebook, Instagram, and TikTok – have been scrutinized for hosting advertisements that overtly sexualize AI avatars.
Companies are advertising adult-focused apps via sexually explicit images and dialogues on social media, some of which involve the sexualization of child-like figures.
According to NBC News, 35 app developers are running sexually explicit ads on social media.
A notable feature of these ads is their tendency to misuse children’s characters, such as SpongeBob SquarePants and the Cookie Monster, to promote adult content.
Most alarmingly, some ads have incorporated AI-generated characters resembling minors, usually rendered in anime style.
This isn’t the first time AI-generated content has been used to distort and manipulate community guidelines and content filters – one TikTok account came under fire recently for generating animations of murdered children and having them recite their fates.
Meta and TikTok said they’re purging these advertisements from their platforms but have refrained from commenting on how the ads evade their content filters.
In all likelihood, content filters are trained on non-AI-generated explicit content, and thus, AI-generated content easily slips through the net.
However, Meta stated that their “policies prohibit ads containing adult content that is overly suggestive or sexually provocative – whether it’s AI-generated or not.”
Who is publishing the apps?
Dozens of unknown app developers are capitalizing on this loophole, running ads that promise “NSFW pics,” “custom pinup girls, “digital girlfriends,” and “chats with ‘no censoring.'”
Many companies are located in countries such as Belarus and China, and apps are generally free to download from Google and Apple’s app stores.
Revenue is primarily generated through in-app purchases and subscriptions.
What’s striking is the inconsistent age rating across these apps: some are restricted for older audiences, while others are deemed appropriate for teenagers.
NBC News tested three of these chatbots. One enabled users to create their own NSFW images, and another was entirely click-based, relying on predetermined explicit storylines.
Among them was the infamous Replika, which was recently linked to an assassination plot to kill Queen Elizabeth II.
Public opinion on the online advertising for these chatbots has been polarized. Comments on Instagram, for example, ranged from “very dystopian” to “just sad,” with some noting that the AI-generated characters looked uncomfortably childlike.
Forthcoming regulation in the EU and US is going to lump pressure on social media companies to crack down on AI-generated content, but whether their filtering technology can deliver the expected outcomes is a different story.
