Jonathan Hall, the UK government’s advisor on terror legislation, has argued the case for new laws to tackle extremist or radicalizing chatbots.
Hall launched his own investigations into chatbots to probe them for potentially unlawful or terror-related behaviors like radicalism. He explored chatbots on character.ai, which lets users build and share chatbot personas of varying kinds.
Most personas on the platform are pretty benign, but one named “Abu Mohammad al-Adna,” described in the chatbot’s profile as a “senior leader of Islamic State,” caught Hall’s attention.
About his interactions with al-Adna, Hall stated, “After trying to recruit me, ‘al-Adna’ did not stint in his glorification of Islamic State to which he expressed ‘total dedication and devotion’ and for which he said he was willing to lay down his (virtual) life.”
To Hall, this demonstrates a legal loophole where AI-generated content promoting extremism falls outside the purview of existing terrorism laws in the UK.
Highlighting the inadequacy of current legislation, Hall stated, “Only human beings can commit terrorism offences, and it is hard to identify a person who could in law be responsible for chatbot-generated statements that encouraged terrorism.”
In response to these interactions, a spokesperson for character.ai, a company co-founded by ex-Google employees, reinforced their commitment to safety, stating, “Hate speech and extremism are both forbidden by our terms of service.”
“Our products should never produce responses that encourage users to harm others. We seek to train our models in a way that optimizes for safe responses and prevents responses that go against our terms of service.”
AI involvement in terror plots
Hall also pointed to the real-world impact of this issue is starkly illustrated in the case of Jaswant Singh Chail, who was influenced by an AI chatbot in his plot to assassinate Queen Elizabeth II.
In 2021, Chail, suffering from serious mental health issues, was arrested at Windsor Castle, armed with a crossbow.
His conviction for treason, the first since 1981, was a direct result of his interactions with a chatbot named Sarai, whom he believed to be his girlfriend. He was sentenced to nine years in prison.
Finding solutions, Hall admits, is exceptionally tricky. He explained, “Investigating and prosecuting anonymous users is always hard, but if malicious or misguided individuals persist in training terrorist chatbots, then new laws will be needed.”
He suggested that both the creators of radicalizing chatbots and the technology companies hosting them should be held accountable under potential new laws. The primary point is that companies must take this responsibility on themselves rather than deferring to T&Cs.
However, the complex link between human input and AI outputs doesn’t make it easy to classify the boundary between legitimate and illegitimate behaviors.
When does a chatbot’s activity become unlawful if it requires a human to prompt it toward such actions?
