AI-controlled autonomous drone defeats human world champion in drone race

A research team at the University of Zurich and Intel reports that the racing drone autopilot AI `` Swift '' raced with three world champions of the drone race and won. There have been examples of AI winning in board games and video games, but the research team says that this is the first time AI has beaten humans in physical sports.

Champion-level drone racing using deep reinforcement learning | Nature

https://www.news.uzh.ch/en/articles/media/2023/Drone-race.html

This time, AI and humans played against each other in the 'FPV drone race', in which the drones fly on a specified course while checking the image of the camera mounted on the drone with special goggles. Until now, autonomous drones had to use an external position-tracking system to precisely control the drone's trajectory in order to fly on a racing course, the researchers said, roughly twice as fast as human-piloted drones. It also took a long time. However, the AI Swift, which challenged the world champion this time, reacts in real time to the data collected by the camera mounted on the drone and grasps the speed and position of the drone, just like a human being uses special goggles. corresponds to the race.

You can see how AI actually competes against the human world champion in a drone race in the following movie.

Champion-level Drone Racing using Deep Reinforcement Learning (Nature, 2023) - YouTube


Swift analyzes images captured by the drone's inertial measurement unit (IMU) and camera.

VIO determines acceleration and position from the flow of surrounding images.

Gate Detector recognizes the gates that make up the course of a drone race. In a drone race, you must pass through gates in a specified order.

In the drone race, which competes for the speed of clearing the course, the posture of the drone changes greatly depending on the situation, so the drone can pass through the gate while completely flipped upside down, or tilt the drone 90 degrees and draw a sharp turn while passing through the gate. is needed. Therefore, Swift recognizes the four corners of the gate separately and adjusts the posture and angle of passing through the gate, including the tilt of its own drone.

A series of analysis results are processed by a neural network, and Swift performs reinforcement learning. The learned results are reflected in the flight of the drone in real time, and the data obtained during the flight is processed again by the neural network. For humans, it's like actually flying a drone and gradually getting the hang of it.

The following three people competed against the drone this time. All of them are fierce fighters who have won the title of champion. The AI will compete against humans after a month-long flight simulation.

The race took place from June 5th to 13th, 2022 in a hangar at Dubendorf Airport near Zurich. In the race, participants pass through seven square gates in a 25m x 25m area in the correct order. Advanced techniques are required, such as reversing the drone's attitude and turning it upside down at full speed.

The difference between Swift and humans was already visible at the start of the race. Swift wasted no time in accelerating the drone as soon as the start signal was given, and Swift got off to a faster start by just 0.25 seconds.

The human-operated drone was truly a champion, and it kept up with the drone operated by Swift while maintaining the small lead it had at the start.

Image seen from the camera of a drone operated by a human. What you can see at the bottom is the leading Swift-operated drone.

However, the champion's drone could not overtake him, and in the end, the drone piloted by Swift won.

Swift optimizes your course by understanding your position and acceleration in real time. As a result, compared to the champion's trajectory (red), the Swift's trajectory (blue) was able to attack the curves more tightly, leading to a reduction in time.

However, the research team argues that ``human pilots were more adaptable than Swift.'' Human pilots can quickly adapt to new courses and fly drones, but Swift sometimes failed when conditions differed from those for which he was trained, such as when the lighting conditions in the room changed. .

The research team claims that the importance of the results of this study goes beyond drone racing. 'There is a limit to the battery capacity of drones. AI will increase the usefulness of drones by allowing them to fly faster and more efficiently,' he said, adding that they can be used to shoot action scenes for movies, save lives in disasters, monitor forests, etc. Autonomous drones are said to be useful in a wide variety of applications, including space exploration.