Warning that there are cases in which ``AI detector'' has destroyed innocent writers' lives by making them falsely accused of cheating

Generating AI such as ChatGPT can create sentences and images that can only be seen as being created by a human. On the other hand, Stanford University andOpenAI , the developer of ChatGPT, have released tools to detect whether text or images are created by AI, and the New York Times has reported on the results of testing multiple AI image detection tools. Many detection tools have also appeared that can distinguish between 'AI or human', which is difficult to judge manually. However, Eric Hauck, who runs the creator support site Authory, has reported on cases in which, as a result of relying on detection tools, the company was accused of misconduct and even got fired.

How AI detectors can destroy innocent writers' livelihoods
https://authory.com/blog/how-ai-detectors-are-destroying-livelihoods

Generative AI can produce extremely high-quality sentences and images, but there are cases where it is not desirable to use AI, such as in creative fields that require original works or in the writing of papers. A paper detection tool developed by a research team at the University of Kansas is said to be able to identify papers written by AI with 99% accuracy.

A tool will be developed that can detect scientific papers written with ChatGPT with more than 99% accuracy - GIGAZINE

On the other hand, the inaccuracy of AI detection tools can also be a problem. During the large-scale conflict caused by the attack on Israel by the Islamic fundamentalist organization Hamas, many fake photos and videos are said to be circulating on social media, and AI detection tools were used. Among them, a user pointed out that an image posted by the Israeli government on social media that conveyed the tragedy of the conflict was ``turned out to be a fake by AI when it was run through an AI detection tool.'' However, experts in digitally manipulated image detection claimed that there was no evidence that the image was generated by AI.

AI-generated image detection system incorrectly identifies photos of the Israeli-Palestinian war as ``fake'', experts point out that they are not generated by AI - GIGAZINE

Mr. Hauck introduces a case in which writer Michael Berben (pseudonym) was seriously damaged by an AI detection tool. According to Hauck, Barben has been a freelancer for three years and has written about 200 articles for various publications. However, one day, a major contributor suddenly told him, ``As a result of starting to use an AI detection tool, we have determined that there is a 95% chance that your article was created by AI.'' In addition, Mr. Barben was informed that all of the articles he had contributed so far had a ``65-95% chance of being created by AI,'' and his contract was immediately terminated.

According to Barben, many of the articles judged to have a ``65-95% chance of being created by AI'' were written before ChatGPT was released. Barben also did everything he could to claim that the writing was done by himself, not by an AI, and made his client's complete history in Google Docs, including the writing process and progress, available. However, the suspicions raised by the AI detection tool were decisive, and Mr. Barben ended up losing his job.

As for the major problem with this matter, Hauck points out, ``The accuracy of common AI detection tools is questionable.'' Mr. Barben's case is one of the most extreme outliers, but false positives are standard in general AI detection tools, and OpenAI released the AI detection tool 'AI classifier' half a year later, saying, 'AI classifier is accurate. 'Currently unavailable due to low availability' and has been suspended from publication.

OpenAI secretly terminates tool to determine whether text is generated by AI due to ``low accuracy'' - GIGAZINE

According to Hauck, it is true that generation AIs such as ChatGPT generate sentences using a specific style, but that can be easily changed with a little tweaking of the prompt. Additionally, even writers who write completely manually have developed a so-called 'ChatGPT-like' writing style. Therefore, ``trying to detect content written by AI based on this fragment is a futile attempt,'' Hauck said.

Second, Hauck points out that figures such as the ``99% accuracy'' that AI detection tools tout are questionable. If you run an AI detection tool on 2000 articles, the AI sentences may be easily distinguishable due to topic or unique style. However, Hauck said, ``Accuracy based on such a small dataset is basically meaningless,'' and that AI detection tools require a sample size of 100,000.

Finally, the biggest problem is that writers have no way to protect themselves from the black box mechanisms of AI detection tools. Proving that the text is written by AI can be done only by the results of the tool, but in order to claim that the text is not written by AI, you must explain the preparation and research required for your work, the writing process, etc. work is required. However, doing so in Mr. Barben's case did not eliminate doubts.