What is the neural network 'OpenAI Five' that challenges the world's top professional gamers?

Nonprofit artificial intelligence that a number of celebrities make donations, such as Ellon Mask of Space X and Tesla , Lead Garrett Hoffmann, founder of LinkedIn, Peter Seal, founder of PayPal AI) Research organization " OpenAI " is an organization promoting R & D to ensure that AI will benefit humanity. Such an OpenAI is focusing on development in recent years is an AI system called " OpenAI Five ".

OpenAI Five
https://openai.com/five/

"OpenAI Five" is an AI system that plays the strategy game " Dota 2 ". Dota 2 is based on a 5 to 5 team battle, but OpenAI Five can operate all five characters of one team at once.

About the question "Why is Dota 2?" OpenAI says "Dota has trafficking in the real world, such as teamwork, long-term perspectives, hidden information, and the real world's awkwardness and continuity to develop a general-purpose AI system that can test bed in the previous year. system that we have used in the training of Dota has shown that the current AI algorithm is capable of executing a large-scale and long-term learning. Also, this system is not used only for learning Dota, it can be used for controlling robot hands, etc. "

By reading the following article you can see what kind of "system that can control fine robot hand" that the same system as this OpenAI Five was used.

A robotic arm showing a movement that is too dexterous like a human hand "Dactyl" - GIGAZINE

OpenAI Five should also be referred to as a team of neural networks and has been learning from the stage where knowledge about Dota 2 is zero. It seems as if he is simulating the human brain as if learning from Dota 2 playing from scratch. OpenAI Five recognizes visual information that human beings can obtain from their eyes as a list of 20,000 numbers and displays eight lists of numbers from among them to determine their next actions. The OpenAI development team is writing code to tie OpenAI Five's numeric list with actions in the game.

Regarding learning of neural networks, OpenAI Five's neural network begins with random parameters and learns better parameters by using Rapid of general-purpose training system. Rapid creates multiple copies of OpenAI Five and can generate gameplay data for 180 years at a stroke by letting each player play games tens of thousands times each day. This parallel processing is comparable to the machine power of 128,000 CPU cores and 256 GPUs.

In each game frame, Rapid calculates the numerical rewards as "positive" when something good happens, "negative" when bad happens. Furthermore, by applying Proximal Policy Optimization of reinforcement learning method, parameters of action created by neural network immediately before positive reward generation are updated. It seems that it reduces the possibility of negative reward generation so that only the action which leads to good results on the game is executed.

Such a step of OpenAI Five starts from a one-on-one battle, not a 5: 5 team battle. The AI ​​I made by OpenAI in August 2017 destroyed Dendi who is the world champion of Dota 2 with a one-on-one battle, you can understand well by reading the following article.

Artificial intelligence of OpenAI development wins human world champion with 1: 1 battle of game Dota 2 - GIGAZINE

After that, based on the knowledge of playing Dota 2 1: 1, the development of OpenAI Five which can play 5 to 5 matches started. Firstly in June 2018 victory against a skilled Dota 2 player with 5 to 5 team battle.

OpenAI artificial intelligence "OpenAI Five" wins the human team with 5 to 5 battle of Dota 2 - GIGAZINE

After that, I announced to play against a prominent player team, I won this match in August 2018.

OpenAI artificial intelligence "OpenAI Five" won against the pro gamer team in a 5: 5 victory - GIGAZINE

A professional gamer team that has won OpenAI Five is not necessarily active as a professional team usually only individuals are professional gamers. So, OpenAI Five sets the final goal of winning against the professional team of Dota 2 who is fighting teams from the usual stage, and the final goal is to hold " The International " which will be held from 20th to 25th August 2018 8 ", that match will be realized at last.

In Dota 2 there is an index named " Matchmaking Rating (MMR)" which is the world ranking of players, but the strength of the opponent of the past OpenAI Five with this MMR is as follows. The horizontal axis shows time and the vertical axis shows the estimated MMR. As of April 2018, the MMR which was about the same degree as the development team as 2000 has a rapid growth of over 6000 lightly as of August 2018.