Nov 25, 2021 16:00:00

Fully automatic driving of Mario Kart 64 using AI succeeded in the actual machine of NINTENDO 64

Using the software library 'TensorFlow ' for machine learning developed by Google, a person who achieved fully automatic running of Mario Kart on NINTENDO 64 has appeared.

Can an AI learn to drive Mario Kart 64? --YouTube


Let's-a Go, TensorFlow! --Hackster.io
https://www.hackster.io/news/let-sa-go-tensorflow-713147d81e18

It was security researcher stack smashing who achieved fully automated driving of Mario Kart. Mr. stacksmashing said that he was impressed with the project 'Training artificial intelligence (AI) to play Mario Kart 64 using TensorFlow' that appeared in 2017, but this project is not an actual machine but an emulator . Since it was done using it, he tried to modify the software so that it could actually run fully automatically using the NINTENDO 64.

The idea of stacksmashing is to take the video output from NINTENDO64 into a PC, perform machine learning, and send the output result from the PC back to NINTENDO64 as a controller input. The AV output from NINTENDO64 was converted to HDMI using an adapter, and the controller signal output from the PC was converted using Raspberry Pi Pico.

The whole picture looks like this. A monitor for observation is also sandwiched between them.

And stacksmashing borrowed 'TensorKart ', a set of machine learning models and tools for controlling Mario Kart 64 created in the above project. Using the PlayStation 4 controller, we let AI learn to relate the frame of the video output from Mario Kart 64 to the input signal of the controller. At this time, adjustments such as reducing the delay and changing the input threshold of the controller were added.

Stacksmashing trained with 10,000 frames, but in the first model the character slammed into the sand or hit a wall.

As a result of continuing to adjust the parameters from there, it was possible to run on the Luigi circuit for all three rounds without problems.

One interesting thing that stacksmashing felt during training was that the AI seemed to be learning from the 'map.' AI learns from the map when the map display is turned on, and learns from the character when the map display is turned off. Furthermore, by eliminating all unnecessary information such as maps and time display, AI performance has improved significantly.

Stacksmashing states that increasing the number of epochs, which refers to the number of times training data was learned, improved performance and completed three rounds with 150 epochs without any problems. The performance was consistent, with a difference in running time between Round 2 and Round 3 of only 0.01 seconds.

After training at the Luigi circuit, stacksmashing

added a new course, the Karakala Desert , to learn. As a result, in this course, he succeeded in completing the course with only two rounds of learning. 'It's possible to run on other courses, but the performance isn't very good. I think we need more training,' said stacksmashing.

In addition, the model 'TensorKartRealHW' improved by stacksmashing is released on GitHub.