Can Gen Z tell AI from human-authored text on Discord?

  • Gen Z individuals, born 1997-2012, make up 36.1% of social media platform Discord’s user base
  • Discord allows bots to be used for messaging, making users vulnerable to AI-generated misinformation
  • A study found that tech-savvy Gen Z users were bad at distinguishing AI messages from human ones

Gen Z individuals, those born between 1997 and 2012, are supposed to be some of the most tech-savvy internet users but they may be more vulnerable to AI misinformation than they think.

Their readiness to adopt new technologies exposes Gen Z internet users to more AI-generated content than many other users.

A study conducted by researchers from Neev Academy, Harvard University, and Harvard Medical School, found that Gen Z isn’t great at distinguishing AI-authored text from human text.

The study ran experiments on the Discord social media platform, which sees 36.1%, or 203.2 million, of its users falling within the Gen Z demographic. Discord is a popular messaging and discussion platform that allows bots to send and receive messages like human users do on its discussion servers.

The researchers put together a group consisting of 335 users of a science, technology, engineering, and math (STEM) education Discord server.

They used GPT-3.5 to generate messages pretending to be a human user describing one of 25 hobbies the user claimed to participate in. For each of the 25 hobbies, a human-authored message was also composed resulting in a total of 50 messages, half AI-authored and half human-authored.

An example of a message sent to participants of the study. Source: arXiv

The messages were sent to the participants on an isolated Discord server using the same bot user to avoid bias. Each participant was shown 10 messages out of the possible 50 with a random amount of human-authored/AI-generated messages. They then had to classify them as either “AI” or “Human”.

The participants were also asked to self-report their experience with AI. The results from the experiment showed that Gen Z individuals identified AI text correctly less than half the time and were even worse at classifying human text as AI-authored.

The participants who claimed the most use of and familiarity with AI performed the worst. This doesn’t bode well for the generation that is most likely to be exposed to AI-generated content.

The results of the study showing how often participants correctly classified AI or human-authored text compared to their self-reported familiarity with AI. Source: arXiv

The researchers found that the average number of correct classifications improved with the age of the participants. Hopefully, that indicates that as we mature we become more skeptical and less likely to be fooled by malicious AI content.

But it also highlights just how vulnerable younger internet users are to misleading AI-generated misinformation.

If the study had been performed using GPT-4 Turbo the line between human and AI would have been even harder to determine. With the anticipation of AGI, none of us is likely to improve on the dismal Gen Z scores.

© 2023 Intelliquence Ltd. All Rights Reserved.

Privacy Policy | Terms and Conditions

×
 
 

FREE PDF EXCLUSIVE
Stay Ahead with DailyAI


 

Sign up for our weekly newsletter and receive exclusive access to DailyAI's Latest eBook: 'Mastering AI Tools: Your 2024 Guide to Enhanced Productivity'.



 
 

*By subscribing to our newsletter you accept our Privacy Policy and our Terms and Conditions