A strange movie that lets you acquire your own skills by making AI compete

It was discovered that artificial intelligence (AI) can acquire physical skills by himself, without explicitly designing the environment with skill in mind. The "physical skills" created independently by AI include various actions such as tackling, faking, kicking, catching and jumping into the ball. Such self play by AI guarantees that the environment for improving AI is always right and will become the core of a powerful AI system in the futureOpenAIHe seems to think.

Competitive self-play
https://blog.openai.com/competitive-self-play/

OpenAI which was established AI as a research institution of the non-profit to open source is, encourage a simple competition for the two 3D robot in the simulator, or going to improve how competitive the contents AI is Was analyzed. Set competition, "Sumo" and a kind of "PK of soccer", one victory be extruded an opponent from the ring, another one of the easy-to-understand rule that victory if touches the ball in the net is.

If the agent performs actions to support exploration such as "standing up and moving forward", you get reward for it. By repeating the competition while repeatedly receiving such simple remuneration, the agent learned actions such as "tackle" "ducking" "fake" "kick" "catch" "diving". Note that the neural network policy of each agent is independentPolicy optimizationIt is said that they are trained individually.

In fact, the skills that AI independently developed can be confirmed in the following movie.

Competitive Self-Play - YouTube


A mysterious 3D model appears on the screen as the movie starts


This is a sumo wrestling rule that one who dropped the opponent from the ring wins


A green model that extrudes opponents with a strange movement


What skills were acquired by competing AI, tackle to the opponent ......



Like a bullfight, I gently cut off my opponent ......



Or taking a stable stance, various.


Another confrontation like a soccer PK that will win if you kick the ball and kick it into the red area.




In this confrontation, using the foot to block the ball ......


Kick the ball in the direction where there is no opponent ... ...




AI independently learned skills such as adjusting the standing position according to the movement of the opponent.



In the game of winning after winning through the opponent and reaching the back line ......


AI created the skill to make an opponent by ducking.



Although it is possible to develop tasks and environments required by each skill, efforts and devises of human designers are necessary, and what kind of skills AI creates depends on each case. However, it is possible to improve the performance of AI by repeated iteration over and over, indeed,Dota 2In the project, we succeeded in creating an AI agent that can beat a top level e-athlete of a human by making AI compete and self-play.