A movie of trying fully automatic running of Mario Kart 64 at Google's machine learning library "TensorFlow"

Open source machine learning software library developed by Google "TensorFlowBy using Mike Kart, we show how a developer reproduced automatic drivingKevin HughesHas opened to you.

TensorKart: self-driving MarioKart with TensorFlow
http://kevinhughes.ca/blog/tensor-kart

Developer Hughes said he was about to play Mario Kart 64 using an artificial neural network during the New Year's holiday free time. In the past few years, Mr. Hughes seems to have thought that "I would like to do something using machine learning", he seems to have come up with this project in particular, trying to use TensorFlow, which is a hot topic recently. The contents of this project is "to let the artificial intelligence learn to play Mario Kart 64 using TensorFlow".

The result can be confirmed in the following movie. The course on which Artificial Intelligence runs is "Luigi Circuit" for beginners even among Mario Kart 64, the driver to use is "Mario", the game mode is "Time Attack".

TensorKart - Luigi Raceway - YouTube


A time attack start immediately.


Start without doing "possible rocket start" by pressing accelerator during countdown before start.


The big curve at the beginning of the course clears without difficulty. Run for some reason to maintain the middle of the three lanes.


Rushed into the tunnel after the curve.


I was running the course cleanly halfway ... ...


I could not bend in the middle of the curve and hit the wall.


Furthermore, I went out of the tunnel and went out of the course once.


However, the big curves after the tunnel have cleared safely ...


Rushed to the second lap.


Although somewhat staggering, we do not go out of the course with a big curve.



However, even on the second lap a little bit collides with the wall inside the tunnel.


Even wandering out of the tunnel with in-course girigiri ... ...


Succeeded to run inside the lane tightly in the place where I just went out for the first lap.


After that, although we clear the big curves without problems with the third lap ... ...


In the curve in the tunnel it will inevitably hit the wall.


And clear as it is. The time is 45 seconds 32 for the first lap, 43 seconds 37 for the second lap, 43 and 97 seconds for the third lap, and 41 seconds 97 for the lap, you can see that the time is shrinking each time you overlap the lap. In addition, dash mushrooms that can be used only three times with time attack were unused, and they did not use any technique such as drift or jump when curving.


In addition, it seems that running a peach circuit is released as a GIF image and running in-course with a curve rather than running on a Luigi circuit. The realization of these is "It was not easy," Hughes wrote in the blog.


In order to make a data set for training, Mr. Hughes self-made a program to take a screen shot of the PC screen and synchronize the image to his computer. And it seems that he first decided which game he plays the game with the emulator of Mario Kart 64 on PC and which screen shot of the position on the PC screen is taken. By using this program, it seems that Mr. Hughes recorded actions taken by "TensorKart" which is an automatic driving artificial intelligence dedicated to Mario Kart 64.


Mr. Hughes modified the data set MNIST of TensorFlow's character recognition tutorial and used software that NVIDIA is developing for an automated driving car to make it simple It is said that it has turned into. And, although it is making machine learning of artificial intelligence, Hughes says "training by TensorFlow was the easiest part in the project." Why is TensorFlow so easy to train, it seems that there are great materials and many tutorials and source code.

Mr. Hughes seems to have no programming experience in C language and he seems to be starting with copying the source code of the original input driver during programming. Hughes' s goal is to reflect the output from artificial intelligence on the emulator of Mario Kart 64 on the PC and successfully run the cart on a real cart as follows by running the cart properly.


Although the environment where TensorKart plays Mario Kart 64 was finished at the end of various preparations, in the first race it seems that the cart ended up in a strange result that it pushed straight into the wall and did not try to bend. In order to fix this, Hughes added a manual driving function to TensorKart. TensorKart made it possible for TensorKart to learn better driving skill by showing Mr. Hughes driving by artificial intelligence when necessary.

Hughes who made various challenges with TensorFlow, "If I do another deep learning project, I probably will not use TensorFlow directly, in that case I probablyKeras(A high level neural network library executable on TensorFlow written in Python) will be used ".

Also, utilizing the same technology used in Google's automatic driving cars, artificial intelligence can train with the majority of simple courses (such as Luigi circuit) with about 20 minutes of training data, training I was able to run even on a course that was not. In addition, he commented, "If there is more data, I can assert that I can also produce complete artificial intelligence for Mario Kart 64."

The source code of TensorKart is published on GitHub.

GitHub - kevinhughes 27 / TensorKart: self - driving MarioKart with TensorFlow