New research has revealed potential bias against non-native English speakers in AI detection software.
The study suggested over half of non-native English human-written essays are being mistakenly flagged as AI-generated. The implications of such false positives are significant, particularly for students and job applicants.
The study tested seven widely used AI text detectors on articles written by non-native English speakers. Results indicated a high false positive rate, with these pieces being erroneously classified as AI-generated.
Stanford University’s assistant professor of biomedical data science, James Zou, led the team that ran 91 essays written by non-native English speakers through seven popular GPT detectors.
These essays, written for the globally acknowledged TOEFL (Test of English as a Foreign Language), had more than half wrongly flagged as AI-generated. In one case, a program even flagged 98% of the essays as AI-created.
Contrastingly, when the software analyzed essays written by native English-speaking eighth graders from the US, over 90% were correctly identified as human-generated.
AI detectors have been widely criticized for penalizing individuals with false positives, though some are considerably worse than others.
One of the study’s key findings is that AI detectors often struggle with accurately interpreting expressions used by non-native English speakers.
These systems have been primarily trained on native English speaker data, leading them to label certain phrases or structures as incorrect simply because they deviate from what is considered native English language usage.
Non-native English speakers face potential discrimination
The implications of this are far-reaching. Non-native English speakers could face challenges when submitting work through automated grading platforms employed by educational institutions.
Additionally, discriminatory algorithms could unfairly penalize students whose first language is not English, perpetuating social inequality in the classroom.
The authors conclude, “We strongly caution against the use of GPT detectors in evaluative or educational settings, particularly when assessing the work of non-native English speakers.”
But why are AI detection algorithms thwarted by non-native English text?
AI detectors rely on “text perplexity,” a measure of how predictably a generative language model can anticipate the next word in a sentence. Low perplexity indicates easy prediction, while high perplexity reflects a less predictable sentence. Complex words, phrases, similes, metaphors, and idioms will ramp up perplexity scores, which non-native English writers may lack.
AIs tend to output a more predictable sequence of words. Thus, humans using common words in familiar patterns risk their work being mistaken for AI-produced text.
Of course, this doesn’t just include non-native written text but also text written in a specific style or reading level. For example, text designed to be highly readable risks being flagged.
Ironically, after identifying this inherent bias, the researchers used ChatGPT to rewrite the flagged TOEFL essays using more complex language.
Once processed again through the AI detectors, all edited essays were labeled human-written. This outcome underlines a somewhat paradoxical situation – these detectors might inadvertently encourage non-native writers to use AI more to escape detection.
Educators are relying on more than AI detectors to clamp down on plagiarism. They’re conducting additional writing tasks in the classroom, looking deeper into reference uses and their accuracy, and analyzing quotes for authenticity.
Meanwhile, educational establishments are establishing rules for AI usage and management, including the UK’s Russell Group of universities, which recently released a joint statement on AI.
