OpenAI is developing technology that can detect text written by ChatGPT with 99.9% accuracy. Is this a countermeasure against students using AI to complete their homework?

With the advent of generative AI, which can output highly accurate answers to various questions, the possibility of students using AI to complete assignments has come into question. To address this issue, OpenAI, which develops the chat AI ChatGPT, has developed a tool that can detect text written using ChatGPT with 99.9% accuracy, according to a report in the Wall Street Journal.

Exclusive | There's a Tool to Catch Students Cheating With ChatGPT. OpenAI Hasn't Released It. - WSJ
https://www.wsj.com/tech/ai/openai-tool-chatgpt-cheating-writing-135b755a

OpenAI says it's taking a 'deliberate approach' to releasing tools that can detect writing from ChatGPT | TechCrunch
https://techcrunch.com/2024/08/04/openai-says-its-taking-a-deliberate-approach-to-releasing-tools-that-can-detect-writing-from-chatgpt/

With the advent of generative AI that can output highly accurate sentences such as ChatGPT, more and more people are leaving sentence creation to AI . It has also been revealed that as of January 2023, approximately 17% of students at Stanford University in the United States ' use ChatGPT for assignments or exams. ' To address this issue, a tool is being developed to detect sentences created by AI .

OpenAI, the developer of ChatGPT, also announced the 'AI Text Classifier', a tool for detecting AI-generated text, in January 2023.

OpenAI releases tool to detect 'text written by AI' - GIGAZINE

However, it has been pointed out that AI-generated text detection tools can mistakenly identify text written by non-native English speakers as 'created by AI'.

Texts written by non-native English speakers are being mistakenly identified as 'created by AI' - GIGAZINE

There have also been reported cases where tools meant to detect AI-generated content have mistakenly identified content by people who are not using AI as AI-generated, resulting in people being falsely accused of fraud.

Warning that 'AI detectors' have destroyed the lives of innocent authors by falsely accusing them of fraud - GIGAZINE

Therefore, OpenAI is developing a technology that is fundamentally different from conventional AI-generated text detection technology, 'detecting sentences created by ChatGPT by adding watermarks to the text,' according to a report in the Wall Street Journal. This can be achieved by making small changes to the way ChatGPT selects words. Basically, it adds an invisible watermark to the output of ChatGPT, which can be detected later by a dedicated tool, and the detection accuracy is said to be very high at 99.9%.

Following the Wall Street Journal report, OpenAI updated a blog post announcing its 'research into technology to detect AI-generated content' and described the text watermarking technology it is developing.

Understanding the source of what we see and hear online | OpenAI
https://openai.com/index/understanding-the-source-of-what-we-see-and-hear-online/

According to OpenAI, text watermarking has been proven to be effective against 'local tampering such as paraphrasing,' which has been employed to detect AI-generated text in the past. However, it is not as robust against tampering that involves using a translation system, paraphrasing with a separate generative model, or asking the model to insert a special character between each word and then remove it.

Therefore, according to OpenAI, it seems that the text watermark can be 'easily circumvented by malicious actors.' In addition, since there are many cases where people who are not native English speakers use ChatGPT for writing work, there is a possibility that such users may misunderstand that the documents they create are 'created by AI from scratch.'

An OpenAI spokesperson confirmed to TechCrunch that they are developing a text watermarking technology that can detect text created using AI, as reported by the Wall Street Journal, with 99.9% accuracy. However, they made it clear that they are cautious about releasing the tool, saying, 'The text watermarking method we are developing is technically promising, but there is a possibility that it could be circumvented by malicious actors or that it could disadvantage groups such as non-English speakers.'