The AI detector misidentifies 1-2% of texts written by human students as AI-made, a terrible accuracy for students who are falsely accused and fail exams.
Since OpenAI's ChatGPT made generative AI practical, 17% of students have used ChatGPT for their assignments, while teachers are also using
Do AI Detectors Work? Students Face False Cheating Accusations - Bloomberg
https://www.bloomberg.com/news/features/2024-10-18/do-ai-detectors-work-students-face-false-cheating-accusations
Below is a graph showing the results of Bloomberg running 500 essays written before the release of ChatGPT through mainstream AI detectorsGPTZero and Copyleaks. The tests determined that three of the 500 essays were generated by AI, and nine were misidentified as having been written in part using AI.
The essays used in the test were submitted to Texas A&M University in the summer of 2022, before ChatGPT was publicly available, so they are unlikely to have been generated by an AI. They were also obtained through a materials request and therefore have never been used to train an AI.
Among the essays that were mistakenly detected as having been generated by AI, some were actually determined to be almost 100% accurate.
Students who are particularly at risk of having their private parts probed when they are not experiencing any symptoms are those with developmental disabilities, who tend to use the same words and phrases over and over, and those learning English as a Second Language (ESL).
A study by Stanford University found that while the AI detector showed near-perfect accuracy in essays written by American-born eighth-graders, it identified more than half of the text written by students whose native language was not English as AI-generated.
OpenAI previously said it was withholding the release of
Moira Olmstead, who has an autism spectrum disorder and tends to write formulaic sentences that can easily be mistaken for AI-generated text, is one person who has been caught up in this problem. After taking time off from college to give birth, Olmstead returned to school to balance raising her child and her dream of becoming a teacher, but an AI detector misidentified her and gave her a zero for one of her required assignments.
When the professor told him about the AI detection tool check, Olmstead said, 'It was like I was punched in the stomach.'
It is no longer clear how much AI has penetrated the educational field. When Bloomberg ran 305 essays submitted in the summer of 2023, after the release of ChatGPT, through the same AI detector as in the aforementioned test, 9% were determined to be AI-generated.
Bloomberg's testing also found that AI detectors could sometimes be fooled by so-called 'AI humanizers,' automated tools designed to disguise AI-generated text as human writing.
Specifically, when an essay that was mistakenly identified by GPTZero as being 'AI-made with 98.1% accuracy' was corrected by an AI humanizer called Hix Bypass, the accuracy dropped to just 5.3%.
Bloomberg points out that this situation risks escalating into an arms race between competing AI technologies, driving a deep rift between educators and students with little educational benefit.
Rather than banning the use of AI, some educators are exploring how to help students use it properly.
Adam Lloyd, an English professor at the University of Maryland who relies on his own instincts to grade assignments rather than using AI detectors and discusses any doubts directly with his students, told Bloomberg, 'Whether we like it or not, artificial intelligence is going to be a part of the future. It would be a mistake to think that we should ban AI from the classroom or encourage students not to use it.'
Instead of banning AI, try to incorporate 'how to use and evaluate' into student education - GIGAZINE
Related Posts:
in Software, Posted by log1l_ks