Fast-paced flight maneuvers mastered with confidence: An aircraft equipped with artificial intelligence has beaten competitors piloted by human champions in drone races. The performance of the system is based on the combination of machine learning capability with physical data collected during the flight. The success is a milestone in the development of mobile robotics with artificial intelligence, the developers say.
They can grasp complex relationships, control robots and even generate clever texts: The sometimes astonishing achievements of artificial intelligence (AI) systems have often been making headlines recently. Because in some cases they can outshine people. This also applies to gaming: AI systems can now easily beat human masters in chess, the tricky thinking game Go or even car races in virtual worlds. However, these are successes in simulation and computer game environments. However, the AI systems have not yet won any gold medals in real competitions that are based on fast and clever control capabilities.
One such game is what is known as first-person-view drone racing. The participants control their quadrocopters at speeds of sometimes over 100 kilometers per hour over a winding route through gates. They wear headsets that allow them to control the drone from a “first-person perspective” through a camera attached to the drone. The champions of this game can steer and accelerate their aircraft so skillfully that it rushes through the racetrack particularly quickly and without collisions.
Developed AI high-speed drone
What human drone pilots achieve was previously considered almost impossible for autonomously flying drones with artificial intelligence. The complex dynamic processes at high speeds are difficult to predict and previous technology reacted too slowly. But as the team of researchers at the University of Zurich now reports, their system called “Swift” can now match human performance and even exceed it under certain conditions.
The racing drone is equipped with sensors for continuous physical data collection as well as an on-board AI system that can be trained and then very quickly ensures “clever” adjustment of the flight controls. Specifically, Swift reacts in real time to the data that comes from a camera and an integrated inertial measuring device that records acceleration and speed. An artificial neural network then takes care of the position and bearing detection as well as the recognition of the gates along the race track. It is connected to another unit of the "brain" of the system that forms the control unit. It then determines the best course of action to complete the challenges of the track as quickly as possible.
Much like its human counterparts, "Swift" needs training to improve its drone piloting skills. This takes place in a simulated environment in which the system teaches itself to fly using the principle of trial and error. This is a form of machine learning known as reinforcement learning. Initial learning in a simulation also helps avoid damaging the drone in the beginning. “To ensure that the consequences of actions in the simulator are as close as possible to those in the real world, we have developed a method for optimizing the simulator with real data,” explains first author Elia Kaufmann from the University of Zurich.
Superior flight maneuvers
After his “training,” Swift was then ready to challenge his human competitors: three human champions, including the world champions of two international leagues. The races took place on a track in a hall with seven square gates, which had to be passed in the correct order through sometimes demanding maneuvers. The human pilots received a week of training on the 0169 circuit after which each competed against Swift in several head-to-head races. The result: The system won multiple races against each of the human champions. Overall, it won 15 of the 25 races and also achieved the fastest recorded race time on the track, half a second ahead of the best human competitor. As the evaluations showed, Swift was apparently capable of somewhat narrower flight maneuvers.
However, so far, humans have only had to admit defeat to AI in drone racing under special conditions, the developers admit: the pilots are still significantly more adaptable. Because the autonomous drone could only demonstrate its performance under exactly the conditions for which it was trained. So far, the system has had problems with changing lighting conditions, less clearly defined gates and it cannot react to disturbances such as wind. But there could also be technical solutions for these challenges in the future.
As the researchers emphasize, the concept is not just a gimmick: “This work represents a milestone for mobile robotics and machine intelligence,” write Kaufmann and his colleagues. There is also concrete potential for use: “Drones have a limited battery capacity – they need most of their energy to stay in the air. If we can fly faster, we can therefore increase their usefulness,” says senior author Davide Scaramuzza from the University of Zurich. “Fast, autonomous drones could also be used in the film industry to record action scenes. And last but not least, high flight speed can make a crucial difference in rescue operations – for example when drones are sent into a burning building,” says the scientist.
Source: University of Zurich, specialist article: Nature, doi: s41586-023-06419-4