Sony announces AI 'Gran Turismo Sophie', skill to overwhelm the world champion of 'Gran Turismo'
Sony
Outracing champion Gran Turismo drivers with deep reinforcement learning | Nature
https://www.nature.com/articles/s41586-021-04357-7
Innovative AI 'Gran Turismo Sophy ™' announced – Sony AI
https://ai.sony/ja/blog/blog-016-jp/
This AI beat the world's best Gran Turismo players | Ars Technica
https://arstechnica.com/cars/2022/02/sony-trains-superhuman-ai-to-race-in-gran-turismo/
GT Sophie was developed in collaboration with Polyphony Digital and Sony Interactive Entertainment (SIE), which develop the Gran Turismo series, under a project that has been underway since the establishment of Sony AI in April 2020. “ Reinforcement learning ” is adopted for learning, which encourages autonomous learning by giving rewards and penalties.
Gran Turismo is a game where you can enjoy car racing, but it also has the aspect of a driving simulator that pursues realism. In racing games, driving techniques that pursue the limit of vehicle performance are required, but in Gran Turismo, the influence of air resistance is also taken into consideration, so it is important how other players are playing.
In addition, tactical points such as the timing of applying the brakes and stepping on the accelerator and pretending to overtake the opponent are also required. In addition, it is very important to follow the rules of the race and try to play fair. In other words, GT Sophie was learning around the three points of 'control,' 'racing skill,' and 'racing etiquette.'
'GT Sophie is different from traditional AI agents like chess, shogi, go, or traditional AI agents like Alpha Star and OpenAI Five for real-time multiplayer strategy video games,' said Sony AI. Instead, it is an AI that handles continuous physical phenomena.
GT Sophie trained in a specific car and course at 'Gran Turismo SPORT'. Some races run on GT Sophie alone, while others race with up to seven computer opponents that appear in regular games. The position of the course, start speed, distance between cars, and skill level of the opponent were set randomly for each race.
The research team rewarded GT Sophie for better performance on the course than last time, and additional rewards for overtaking other cars. On the other hand, he said that he was penalized if he couldn't turn the corner, hit a wall, slipped, or hit another opponent's car.
It took several hours of training to learn how to run the course, but GT Sophie was better than 95% of Gran Turismo SPORT players within a few days. As a result of running for 45,000 hours, he seems to have set a superhuman time on the three courses used in the training.
In addition, when we invited three players who hold the world's top level time in Gran Turismo SPORT and played against GT Sophie, none of them could beat GT Sophie.
For example, GT Sophie showed a racing line on a course called 'Dragon Trail Seaside' that even Emily Jones, who holds the world's top level record, did not notice. 'On the approach to the first corner, I was braking later than GT Sophie, but GT Sophie performed much better and overtook me in the next corner. I didn't notice it until I saw the AI running, and I thought, 'Why, I should do that instead.'
The GT Sophie and Jones races can be seen below.
Gran Turismo Sophy ONE ON ONE # 1 | Emily Jones --YouTube
At the Lake Maggiore Circuit, human Valerio Gallo had a lap time of 114.466 seconds, while GT Sophie had a lap time of 114.249 seconds. Mr. Gallo was able to analyze the running of AI after the race and then extend the lap time to 114.181 seconds.
In July 2021, we invited top-level racers from Gran Turismo SPORT, Takuma Miyazono, Tomoaki Yamanaka, Ryosuke Kokubun, and Shotaro Ryu, to the race. In this race, GT Sophie and humans played against each other with the rule of giving points according to the ranking of goals, and the result was 86 to 70, which was the victory of the human team.
You can see the race between the human team and the GT Sophie team in Tokyo in July 2021 in the following movie.
Gran Turismo Sophy RACE TOGETHER 2021 # 1 [Japanese] --YouTube
However, in the rematch held in October 2021, as a result of changing the training method of GT Sophie, the GT Sophie team won overwhelmingly with 52 to 104. 'The advantage of racing AI training methods with deep reinforcement learning is that agents learn the right behavior through trial and error, as long as the conditions are right,' the research team said.
The following movie is the race held in October 2021.
Gran Turismo Sophy RACE TOGETHER 2021 # 2 [Japanese] --YouTube
However, the research team points out that there is room for improvement in strategic decision making. For example, GT Sophie tries to overtake the opponent as much as possible when the course is straight, but it is often overtaken by braking in the next corner. Also, if there is a car that pits in with a time penalty, it usually waits for the car that received the penalty to slow down, but GT Sophie seems to try hard to overtake.
Sony AI said, 'GT Sophie has achieved an important milestone, but it is still in the early stages of research and development. Sony AI will continue to improve the performance of GT Sophie in partnership with PDI and SIE. , We will consider how to incorporate GT Sophie into the 'Gran Turismo' series. ' It also suggests that players may be able to play against GT Sophie in the game in the future.
The development of Gran Turismo Sophie is also discussed in detail in the following movie released by Sony AI.
The Making of Gran Turismo Sophy [Japanese] --YouTube
Related Posts:
