A tool will be developed that can detect scientific papers written in ChatGPT with an accuracy of 99% or more
In recent years, the development of AI that generates sentences and images has been remarkable, and in a survey conducted by Stanford University, about 17% of students answered that they `
Distinguishing academic science writing from humans or ChatGPT with over 99% accuracy using off-the-shelf machine learning tools: Cell Reports Physical Science
https://doi.org/10.1016/j.xcrp.2023.101426
AI-generated academic science writing can be | EurekAlert!
Scientists claim over 99 percent identification of ChatGPT • The Register
Many traditional tools for distinguishing between human-written and AI-generated text were not built specifically for professional scientific papers. Therefore, the research team of Mr. Desaire et al. has developed a tool that can accurately distinguish humans and AI for scientific papers.
The research team selected 64 papers from various fields such as biology and physics, and generated 128 sentences to explain the same content to ChatGPT. A total of 1276 paragraphs were generated by ChatGPT and used to train the algorithm.
According to the research team, the criterion for determining whether a paper was written by a human or a paper generated by ChatGPT is the ``difference in style'' between the sentences of humans and ChatGPT. Scientists who publish papers often know more specialized words and phrases than ChatGPT, and are likely to write long paragraphs containing a variety of words. It has also been pointed out that human-written papers contain many question marks, parentheses, semicolons, etc. that are not often included in sentences generated by ChatGPT.
Also, in contrast to ChatGPT, where the number of characters and words per paragraph tends to be constant, the number of characters and words per paragraph and the length of sentences in papers written by humans are not constant. Also, in papers written by humans, words such as “but”, “however”, and “because” often appear in papers, while in papers generated by ChatGPT, phrases such as “other” and “researchers” are often used. It has been.
The algorithm was then tested using 30 human-written papers and 60 ChatGPT-generated sentences based on the abstracts of the papers. As a result of the experiment, it is reported that the tool developed by the research team was able to identify human-written papers and AI-generated papers with almost 100% accuracy. Using the tool for each individual paragraph in the paper reduced the accuracy of detection slightly, yielding an accuracy of about 92%. According to the research team, this tool significantly exceeds the accuracy of AI text detection tools available on the market.
As a future goal, the researchers are looking at using the tool in a wider variety of academic papers, and as AI advances to generate more human-like text, their It mentions doing some research to see if the tool will help.
On the other hand, overseas media The Register said, ``Many software designed to detect text generated by AI has a problem of low reliability.'' Nevertheless, it is not clear how accurate our tool will be at detecting lightly human-edited papers, so the results presented by the tool are for reference only. I should accept it,' he said.
What is the accuracy of the tool `` GPTZero '' that distinguishes between humans and AI who wrote sentences? -GIGAZINE
In response to the question, ``Can you use this tool to determine whether a student has written a paper using ChatGPT,'' Deser said, ``This tool can judge AI and professional scientists. While they are good at , they are not designed to judge papers and essays written by students.'
Related Posts:
in Software, Posted by log1r_ut