Is AI risky or not? It’s the debate that just keeps raging on without definitive answers.
Contributing more evidence to this debate, University of Bath and the Technical University of Darmstadt researchers launched a study to evaluate AI risks in the context of current language models like ChatGPT.
The findings, published as part of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024), challenge views that AI, particularly large language models (LLMs) like ChatGPT, could evolve beyond human control and pose an existential threat to humanity.
This confronts the fears expressed by some of the world’s leading AI researchers, including Geoffrey Hinton and Yoshua Bengio, two of the “godfathers of AI” who conveyed concerns about the potential dangers of advanced AI.
Yann LeCun, the third “godfather of AI” and Meta’s chief AI scientist, alongside Dr. Gary Marcus and others, argues the counter – that AI risks are simply overblown.
This divergence in opinion among the field’s most influential figures has fueled a fierce debate about the nature and severity of the risks posed by advanced AI systems.
This new study probes LLMs’ “emergent abilities,” which refer to a model’s ability to perform tasks for which it was not explicitly trained.
AI risks are multifaceted, but at least some relate to models developing their own goals that could harm humans, like shutting down computer systems or leaking data.
The worry under inspection is whether or not an LLM might spontaneously develop these skills without instruction or control.
To investigate this, the research team conducted a series of experiments:
- They examined the underlying mechanisms of “in-context learning” (ICL) in LLMs, which allows models to generate responses based on examples provided during interactions. As the study states, “The ability to follow instructions does not imply having reasoning abilities, and more importantly, it does not imply the possibility of latent, potentially-dangerous abilities.”
- They assessed LLMs’ true capabilities and limitations by evaluating their performance on a range of tasks, including those that require complex reasoning and problem-solving skills. The researchers argue that LLMs can’t independently develop new skills
- They analyzed the relationship between model size, training data, and emergent abilities to determine whether increasing model complexity leads to AI developing hazardous skills. The study said, “These observations imply that our findings hold true for any model which exhibits a propensity for hallucination or requires prompt engineering, including those with greater complexity, regardless of scale or number of modalities, such as GPT-4.”
The researchers conclude from their investigation that “the prevailing narrative that this type of AI is a threat to humanity prevents the widespread adoption and development of these technologies and also diverts attention from the genuine issues that require our focus.”
This strongly aligns with LeCun and others who believe AI risks are over-publicized.
However, while evaluating the risks posed by current AI models is obviously essential, accounting for the future is a tougher task.
Each generation of models comes with new abilities and, thus, new risks, as shown by some strange behaviors documented in GPT-4o’s test card.
One red teaming exercise (designed to identify unpredictable AI behaviors) quite shockingly saw GPT-4o’s voice feature unexpectedly clone a user’s voice and start talking to them in their own voice.
Tracking AI risks as and when they emerge is critical, as the goalposts are changing all the time.
The study makes a salient point that some non-existential AI risks are already knocking at the door, “Future research should therefore focus on other risks posed by the models, such as their potential to be used to generate fake news.”
As the authors admit, then, just because AI doesn’t pose large-scale threats right now doesn’t mean safety is a non-issue.
