Benchmark to evaluate AI in boss battle of `` Rockman 2 Dr. Wiley's Mystery '' appears

by -EMR-

With the evolution of AI (Artificial Intelligence) algorithms, AI can play computer games originally designed for humans to play, and even beat the world ranker and have strength equivalent to the top 0.2% of humankind boasts also appeared thing. Using Capcom's horizontal scrolling action ' Rockman 2 Dr. Wiley's Mystery ' released on Family Computer in 1988, a benchmark ' EvoMan ' for evaluating AI performance was developed.

[1912.10445] EvoMan: Game-playing Competition
https://arxiv.org/abs/1912.10445

EvoMan – Karine Miras
https://karinemirasblog.wordpress.com/portfolio/evoman/

Researchers want to use Mega Man 2 to evaluate AI | VentureBeat
https://venturebeat.com/2019/12/26/evoman-mega-man-ii-capcom-ai-benchmark/

There is a precedent for research that uses games to evaluate the performance of AI algorithms. For example, the Arcade Learning Environment , announced in 2012, is a benchmark designed to test a wide range of AI technologies, using over 55 Atari 2600 games. In addition, OpenAI, a non-profit organization that researches artificial intelligence, announced the Procgen Benchmark, which evaluates sample efficiency and generalization performance in reinforcement learning in 16 types of games.

The research team of AI researcher Fabricio Olivetti de Franca and others has announced ' EvoMan ', which uses 'Rockman 2 Dr. Wiley's Mystery' to evaluate the generalization of AI algorithms and the performance improvement by co-evolution.

'Rockman 2 Dr. Wiley's Mystery' consists of eight selection stages and six Wiley stages. Boss is set in each of the eight selection stages, and you can get new weapons by defeating the boss. Each boss has its own weaknesses, so thinking about which weapon to defeat which boss is the key to a game strategy.

In EvoMan, Rockman controlled by AI will capture 8 boss battles. Agents learn how to identify and respond to enemy attack and avoidance patterns in any of the four boss battles. Then, EvoMan measured the AI adaptation strategy of 'how far the learned strategy can pass to the remaining four bosses'. Ultimately, the AI algorithm will be ranked based on the average of the performance in the eight boss battles.

The following movie shows how AI of a neural network (NEAT) that actually learns EvoMan with a genetic algorithm . The appearance of EvoMan is completely different from the original family 'Rockman 2 Dr. Wiley's Mystery', but the movement and attack pattern of the boss (yellow character) is the same.

EvoMan-MultiEvolution-NEAT-YouTube


In addition, the research team encouraged AI to evolve by fighting against 'AI-controlled rockman' versus 'AI-operated boss' to verify that each algorithm was capable of defeating eight bosses. Eventually it was said that there were four Airman Metalman Quickman Bubbleman that could be defeated by evolution. Researchers argue that through this research, the easiest boss to defeat in Rockman 2 Dr. Wiley's Mystery is Airman.

EvoMan-Coevolution-YouTube


Regarding why they developed an AI benchmark using `` Rockman 2 Dr. Wiley's Mystery '', the research team said, `` How general is AI to win against one enemy or to crush an enemy rushing, 'Rockman 2 Dr. Wiley's Mystery' is a good benchmark to test how much you can co-evolve to create a difficult enemy. '