Stolen faces, stolen lives: The disturbing trend of AI-powered exploitation

april 18, 2025

  • Fake AI-generated influencers are taking over social media
  • Among the most insidious are fake influencers with Downs syndrome
  • People are creating and monetizing extreme forms of AI pornography

Most social users will have come across an influencer who looks a little… off. 

Maybe their facial features are a bit too symmetrical, and their poses are a bit too rigid. Chances are, you’re not looking at a human at all – but an AI-generated forgery.

In some cases, these AI influencers are reasonably benign – just digital forms of their real counterparts but not overtly trying to deceive or manipulate. 

However, this isn’t always the case. Disturbingly, there’s a network of Instagram accounts using AI to create fake influencers with Down syndrome. The goal? To exploit a vulnerable community for likes, shares, and ultimately, cash. 

But the deception doesn’t end there. Many of these accounts link out to shady adult websites, where the AI-generated content is monetized. 

Sadly, this is just the latest evolution of the “AI pimping” trend, where unscrupulous operators use machine learning to create counterfeit influencers for monetary gain. It’s not just Down syndrome, but fake amputee models, burn victims, and other forms of AI-generated pornography.

AI image and video models are now approaching a level of realism that makes them very viable substitutes for real humans. It’s affecting the fashion industry – where real models are facing replacement at the hands of the AI clones. 

Even household names like H&M are wading into these murky waters. The fast fashion giant recently announced a campaign featuring AI-generated “digital twins” of real models. Back in 2023, a company called lalaland.ai released tools for creating AI models for a subscription fee. 

While H&M insists the models maintain control over their digital likenesses, many in the industry are skeptical. After all, in an era of cost-cutting and consolidation, why hire human talent when you can license a cheap, infinitely replicable digital avatar?

The latest, most insidious twist here concerns the fundamental dignity and humanity of marginalized communities. 

People with Down syndrome – or any disability – are not props to be manipulated for profit. 

Moreover, the proliferation of AI-generated content threatens to erode public trust in media altogether. If we cannot trust the images we see online, the very foundation of digital discourse starts to erode.

So next time you’re scrolling through your feed and an influencer seems too good to be true, trust your gut. 

Deltag i fremtiden


TILMELD DIG I DAG

Klar, kortfattet, omfattende. Få styr på AI-udviklingen med DailyAI

Sam Jeans

Sam er videnskabs- og teknologiforfatter og har arbejdet i forskellige AI-startups. Når han ikke skriver, kan han finde på at læse medicinske tidsskrifter eller grave i kasser med vinylplader.

×

GRATIS PDF EKSKLUSIVT
Vær på forkant med DailyAI

Tilmeld dig vores ugentlige nyhedsbrev og få eksklusiv adgang til DailyAI's seneste e-bog: 'Mastering AI Tools: Din 2024-guide til forbedret produktivitet'.

*Ved at tilmelde dig vores nyhedsbrev accepterer du vores Politik for beskyttelse af personlige oplysninger og vores Vilkår og betingelser