The probability that humans can identify `` sentences written by AI '' is about 50%, which is only the same level as coin toss
Large-scale language models (LLM) such as OpenAI's
Human heuristics for AI-generated language are flawed | PNAS
https://doi.org/10.1073/pnas.2208839120
Was this written by a human or AI?
https://hai.stanford.edu/news/was-written-human-or-ai-tsu
AI-generated texts are becoming more and more embedded in our daily lives. For example, Microsoft's Bing is equipped with ChatGPT, making it possible to search for information while interacting with humans.
Microsoft announces new search engine Bing and browser Edge integrating ChatGPT's upgraded AI - GIGAZINE
Also, according to a survey by security company McAfee , 31% of adults said they plan to use AI for their dating app profiles or are already using it.
A research team led by Jeff Hancock, a communication professor at the Graduate School of Humanities at Stanford University and a researcher at HAI, uses the dating app OKCupid, the vacation rental app Airbnb, and the job search app Guru.com to help humans ' We investigated how accurately we could identify whether a text was written by a human or a text written by an AI.
4,600 subjects were presented with text samples from OKCupid, Airbnb, and Guru.com, and determined whether the presented text was human-generated or AI-generated. As a result, it was found that humans could only identify the author of the sentence with an accuracy of 50 to 52%. This is about the same probability as judging the flip side of a tossed coin.
We found that it is difficult for humans to distinguish between sentences written by AI and sentences written by humans, but on the other hand, it turned out that humans do not make decisions based on mere guesswork. For example, when it has the feature of ``accurate grammar and uses first-person pronouns'', there were cases where it was misunderstood as written by humans. Other references to family life and casual colloquial language were also misinterpreted as human writing.
``We all relied on similarly flawed heuristics ,'' said co-author Maurice Yakesh.
“The real concern is being able to make AI that looks more like a human than a human,” Hancock said, “because it can be optimized to take advantage of assumptions that humans have, and we can It creates the risk of deceiving people by making them feel more human than us.'
Furthermore, Mr. Hancock said, 'The amount of content generated by AI may overtake human-generated content in just a few years, and our information system may really collapse. , the underlying trust may be lost and trust in each other may decline.'
The research team proposes to add habits to AI sentences as an idea to distinguish between human and AI text. For example, even in the case of humans, even if they speak the same language, there are dialects depending on the region where they were born and raised, and the use of words and accents differ, so it is often possible to tell where a person is from by the way they speak. The idea is to apply this and introduce a unique wording to the AI as a digital watermark to the AI's text, making it easier to identify the artifacts of the AI.
“We often look at technology, but we ignore the fact that it is we humans who use and adapt to it,” Hancock said. I hope that we can figure out how to use it, create norms, and even create policies and laws that limit its negative use.'
Related Posts:
