AI can reproduce Super Mario Bros. and Rockman's game itself from game play videos

So far artificial intelligence (AI)Win the world's strongest players at Go gameOrPlay a simple game better like a breakout or a space invader better than a humanWe have done various results. All of these are the result of letting AI teach "How to play the game?" The research group of the Georgia Institute of Technology is pursuing AI research with a completely different approach, rather than "How to play?" The AI ​​system which can reproduce the game itself is created by learning the mechanism itself.

AI learns to re-create Super Mario Bros. by watching someone else play it - The Verge
https://www.theverge.com/2017/9/10/16276528/ai-video-games-game-engine


The research group of the Georgia Institute of Technology published "(PDF)Game Engine Learning from Video(Learning the game engine from the movie) "is described about the AI ​​system which can reproduce the game engine of the game like Super Mario Brothers. The AI ​​system developed by the research group seems to be able to learn about the game engine just by looking at the pixels of the game screen and create similar game engines without accessing the game code.

However, it does not mean that you can create exactly the same game from scratch by just showing the game screen. The AI ​​system that imitates the game engine provides two important pieces of information before learning. One is data of all characters appearing in the game, and the other is a basic concept set on the position and speed of the object. Based on these two data, the AI ​​system seems to disassemble the game play movie for each frame, classify the elements to be displayed, and unravel the rules in the game action.

The following GIF images are gameplay screens of the usual Rockman on the left and Rockman's game screen reproduced on the right based on the game play movie by AI. There are strange things such as the object pillar on the screen disappearing or the floor moving finely, but you can see that the movements of Rockman and enemies are almost exactly reproduced. However, it is lovely that the lock buster is not straight but fly obliquely through the floor.


"(Game play) For each frame of the movie we have a parser (analysis tool) and we use it to gather information, for example what kind of information is gathered, what kind of animation state It is in what speed and the object is moving, "said Matthew Guzdial, the first author of the paper.

In addition, Mr. Guzdial said, "Imagine Mario is just above Kuribo in one frame and imagine that Kuribo will disappear in the next frame." In this case, "Mario is right above Kuribo and the speed is downward In some cases, I notice that there is a rule that Kuribo goes out (to be knocked down) ", I will tell you how AI will unravel rules in the game by analyzing the play movie of the game .

With the passage of time, the AI ​​system builds every detailed rule, records it as a series of logical statements, and seems to bring it closer to the game engine by combining these. The detailed rule analyzed by the game can also be exported to the programming language used when AI reproduces the game itself.

ByWOCinTech Chat

However, in the current AI system, there is a limit to reproducing games on the 2D platform, as it depends on humans to define what happens in a particular game. In the case of 3D games, more advanced machine vision tools and analysis time are required to define all information.

In addition, the researcher group at the Georgia Institute of Technology not only unveils this technology to the video game system in the future, but also thinks that it will be useful in real life. As human beings understand, it is obvious that several breakthroughs are required to realize, as AI will also make efforts to make the world understand. But it is not impossible, "In future versions I think we will be able to analyze the limited reality area," Guzdial says.