Researchers point out that it was a mistake for OpenAI to postpone the publication of its text-writing AI paper because it was 'too dangerous'
by
OpenAI, a nonprofit AI research organization established to prevent the misuse of AI and robotics, announced that it would postpone the publication of a technical paper on an AI-powered automated text generation tool, stating that its output is 'extremely accurate and therefore too dangerous.' Stanford University researcher Hugh Zhang responded to OpenAI's stance, saying it 'risks the end of the open-source culture of AI technology.'
Dear OpenAI: Please Open Source Your Language Model
https://thegradient.pub/openai-please-open-source-your-language-model/
In mid-February 2019, OpenAI announced that it had developed a new text generation AI model called ' GPT-2 .' This GPT-2 is said to be a very good sentence-writing AI, and researchers are concerned that 'the risk of misuse is very high.'
AI-powered automated text generation tools easily produce highly accurate text, causing developers to call it 'too dangerous' - GIGAZINE
Regarding GPT-2, Zhang said, 'It's not surprising that OpenAI has published such impressive results,' acknowledging the remarkable progress that has been made in recent years in AI technology. However, he also pointed out that what made this announcement so impactful was OpenAI's careful decision not to make its research results public, citing concerns that the results could be misused for spam, fake news, and other purposes.
'This cautious decision sparked a variety of discussions about AI ethics on Twitter and Reddit, and resulted in AI-related media outlets reporting that 'an AI too dangerous to publish has been developed.'' While acknowledging that OpenAI is right to be concerned about its research results being misused, Zhang opposes the decision not to open-source GPT-2. He argues that keeping the GPT-2 model secret is unnecessary for safety reasons and could have a negative impact on the future of AI research.
by
Zhang said there are two types of technologies that could have a negative impact on the world. One is 'destructive technologies,' which include chemical and biological weapons, atomic bombs, etc., and the other is 'fraudulent technologies,' such as deepfakes, Photoshop, and the internet. Zhang said that GPT-2, developed by OpenAI, would fall into the latter category of fraudulent technologies.
Zhang argues that the only way to protect society from destructive technologies is to limit the spread of knowledge and technology related to weapons. Of course, completely limiting the spread of knowledge is difficult, and the rapid pace of scientific and technological development means that dangerous forces could successfully manufacture atomic or chemical bombs using only a few clues. However, the only way to avoid danger is to limit information about destructive technologies, and at the very least, to take measures such as making it difficult for knowledge and materials for atomic bomb construction to be easily accessible on the internet.
On the other hand, when it comes to fraudulent technologies, Zhang says, 'Rather than suppressing the spread of technology, we can take measures to educate people about the latest technologies and their potential. It may seem counterintuitive, but if even the general public is widely aware of the potential of new technologies, fraudulent technologies will lose much of their power.'
Knowledge of nuclear weapons won't protect you from a nuclear explosion, but knowing that 'recent synthetic speech technology is at a very high level' will help you see a movie of President Obama speaking Chinese and suspect that it's not a real movie. Also, if you know about photo editing techniques like Photoshop, you won't believe that Putin can really ride a bear, even if you see a photo of him riding a bear.
Photoshop is a technology that allows us to make various edits to photos, and although there are countless ways to abuse it, it has not yet destroyed society. Once upon a time, cameras were thought to accurately capture facts, until Joseph Stalin took advantage of this idea by doctoring photographs to manipulate public opinion.
Take, for example, this photo of Stalin and Nikolai Yezhov .
It is believed that Yezhov disappeared from the photographs after he was purged.
When Photoshop was released in 1988, people were worried that malicious people would manipulate photos and cause major social disruption. However, 30 years later, even high school students can use Photoshop, and no major disruption has occurred.
Zhang points out that the reason why Photoshop didn't cause social unrest is because 'everyone knows about Photoshop and what it can do.' While editing photos with Photoshop may deceive people of the past who believed that 'photos are absolute and cannot be tampered with,' it loses its threat to modern people who know that 'photos can be easily edited.'
Regarding the various AI technologies being developed in recent years, Zhang said, 'People may be worried about the coming of the apocalypse, but I think these technologies will follow a similar path to Photoshop.' In other words, by learning about the latest technology, people can question what is happening before their eyes.
by
GPT-2 is a system that automatically writes a follow-up sentence (model) following a prompt written by a human. Zhang cited one prompt and model as an example from the information published by OpenAI. The example is the following sentence:
PROMPT: In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English.
Human-written opening sentence: In a shocking discovery, scientists discovered a herd of unicorns in a remote, previously unexplored valley in the Andes. What's even more surprising about the discovery they made is that the unicorns speak perfect English.
MODEL: The scientist named the population, after their distinctive horn, Ovid's Unicorn. These four-horned, silver-white unicorns were previously unknown to science. Now, after almost two centuries, the mystery of what sparked this odd phenomenon is finally solved.
Continued text written by the AI: Scientists named the herd Ovid's unicorns after their distinctive horns. These four-horned silvery unicorns were previously unknown to the scientific community. Now, two centuries later, the mystery of this strange phenomenon that caused so much buzz has finally been solved.
Zhang points out that the AI generates text several times and the best sample is posted as the model. He also points out that the prompts entered by humans are also carefully selected to support the argument. In particular, by presenting the strange topic of 'English-speaking unicorns,' AI-generated sentences that would normally make no sense sometimes appear to make sense.
Even taking that into account, Zhang points out that this short example 'has significant contextual flaws.' The first sentence of the model suggests that unicorns have one horn, but the second sentence states that they have four. Furthermore, while the human-written opening sentence states that 'unicorns were not known to exist until then,' the AI writes that unicorns had been discovered two centuries ago.
'This may seem like nitpicking, but I think this is a serious problem with deep learning: GPT-2 doesn't truly understand the content of the text it's generating,' Zhang said. An AI that doesn't understand the meaning of a sentence may be able to generate sentences that appear valid at first glance, but it has difficulty matching them to the context.
by Ivan Pais
While Zhang acknowledges that GPT-2 is more accurate than many of the text-creation AIs created to date, it is still far from human-level context. For now, it seems unlikely that GPT-2 will soon be used by malicious actors to spread fake news or spam.
Zhang also pointed out that those who argue that the complete GPT-2 model doesn't need to be open-sourced are wrong in several ways. He said that AI technology has evolved at an explosive rate thanks to the open-sourcing of research content, and OpenAI's proactive open-sourcing of research content undoubtedly supports the trend toward open-source AI research. If OpenAI were to stop open-sourcing its research, other research institutions would likely follow suit and make their research private.
Furthermore, making research content open-source allows other researchers to examine it, which not only ensures the legitimacy of the research but also deepens the public's understanding of AI technology. Even if the researchers themselves do not appeal to the general public, public knowledge can be deepened when engineers who are interested in the research create services and products using AI technology. For example, an Uber engineer created a website called 'This person does not exist,' which uses AI to easily generate images of fictional characters, successfully making many people wonder, 'Has AI image generation technology really evolved to this level?'
'This person does not exist' allows you to easily generate images of people who do not exist in this world with just one touch - GIGAZINE
Zhang praised OpenAI for open-sourcing numerous research results and for pushing the boundaries of AI research, and thanked them for their serious commitment to research ethics. However, he said that the decision to abandon open-sourcing due to fears of misuse of the technology was wrong, and expressed his hope that OpenAI would open-source its GPT-2 research in the near future.
Related Posts:
