India’s 2024 general election, the largest democratic exercise in the world with over 960 million eligible voters, is being transformed by the rise of AI intelligence and hyper-realistic deep fake content.
With this, a new generation of tech-savvy political operatives is leveraging AI to create synthetic media with political and commercial intent.
One of the most prominent figures in India’s emerging ‘deep fake industry’ is Divyendra Singh Jadoun, a 31-year-old self-styled “Indian Deep faker” who has quickly become a go-to provider for political campaigns.
What started as a solo operation making Bollywood spoofs has evolved into a full-fledged synthetic media studio.
“He’s since expanded because of all of the work that is coming to him,” Bangalore-based reporter Saritha Rai tells Bloomberg.
“The first time I spoke to him, he said he had a couple of workers, and that was a few months ago. Then, he told me he has five other employees who help them make these videos.”
Using AI models trained on hours of authentic footage, Jadoun and his team generate uncannily realistic videos of politicians like Prime Minister Narendra Modi appearing to address voters by name.
“The level of hyper-personalization that is possible with generative AI…You can just train the model to say the main message and then feed it with thousands and thousands of names. And it will instantly reproduce that same video and sync it perfectly with Narendra Modi calling out each person by name,” explains Rai.
AI election campaigns mimic the intimacy of in-person interactions at an almost infinite scale and affordable cost.
Rai further notes, “a lot of the campaigning is going to be hyper personalized to the electors…a lot of it is going to reach them via their phones. And a lot of it will be AI generated material.”
Deep fakes are rife in India ahead of the election
Compounding the issue in India, Rai explains how millions of people are introduced to technology for the first time through affordable smartphones.
A digitally inexperienced population is far more susceptible to deceptive content.
The same can also be said of older people, who, again, aren’t so exposed to the foolery of internet content.
But even among those who rate themselves as effective at identifying AI fakes, studies reveal that even when we know media is fake, it can affect our memory and decision-making.
In addition to deep fakes designed to paint public figures in a positive light, instances of malicious political deep fakes are also becoming more common in India.
For example, in November, fact-checker Muralikrishnan Chinnadurai spotted a livestream purporting to show Duwaraka, the long-dead daughter of a Tamil militant leader, giving a fiery political speech.
Chinnadurai identified glitches, revealing it to be an AI-generated avatar. “This is an emotive issue in the state [Tamil Nadu] and with elections around the corner, the misinformation could quickly spread,” he told the BBC.
Last month, deep fake videos of Bollywood stars Ranveer Singh and Aamir Khan seemingly endorsing the opposition Congress party went viral, prompting police complaints from the actors.
On April 29th, Prime Minister Modi himself warned about deep fakes being used to doctor speeches by ruling party leaders.
The next day, two opposition party workers were arrested for a forged video of Home Minister Amit Shah. The opposing Bharatiya Janata Party (BJP) has faced similar accusations from rivals.
So far, India’s government has offered scant oversight, largely outsourcing content moderation to social media platforms. “India has no regulation at all, when it comes to AI,” states Rai.
The government recently began requiring prior approval before launching “unreliable” AI tools and warned against outputs that could undermine elections, but comprehensive rules are lacking. The same can be said for much of the world.
As the former election commissioner, SY Qureshi aptly warns, “Rumours have always been a part of electioneering. [But] in the age of social media, it can spread like wildfire…It can actually set the country on fire.”
