They are already equipped with a keen eye - but unfortunately rescue drones have so far been deaf to the often quiet cries for help from disaster victims. Fraunhofer researchers now want to change that: They are developing a sophisticated hearing system for drones that can precisely locate the sounds of buried people through directed perception and recognize them using artificial intelligence. Drones equipped with “fine ears” will be able to guide rescue workers quickly and precisely to locations in the future, say the scientists.
After natural disasters or bomb attacks, every minute counts when it comes to tracking down victims in the affected areas and providing them with help. Drones have already established themselves as important tools: They can quickly rush into areas that are often difficult to access and provide an initial overview of the situation and help track down victims. High-resolution daylight and thermal imaging cameras help you with this. But victims trapped under rubble remain hidden from these systems and visual capabilities are limited even in thick smoke, fog and darkness.
However, buried victims can also make themselves known by shouting, knocking or clapping. So far, however, it has proven difficult to equip the buzzing drones with an effective ability to hear these sometimes subtle noises. Now a team of researchers from the Fraunhofer Institute for Communication, Information Processing and Ergonomics (FKIE) offers a technical solution to this challenge. Their acoustic supplement to the vision of drones is called “Listening system Using a Crow's nest arraY”, or “LUCY” for short.
Sophisticated localization of noises
The sensory unit consists of a structure with a tree structure: 48 microphones are attached to an array with a special geometric arrangement so that sound can be recorded from all directions. These are small MEMS microphones. They are robust and cost-effective as they are also used in smartphones, for example. The array is small and light and can therefore be easily mounted on conventional drones. Because of its ability to provide an acoustic overview, the scientists call the structure a crow's nest array. “The highest vantage point on ships is called Crow's Nest, from there you can see in all directions. This also applies to LUCY – our system can hear virtually unrestricted in all directions,” says Macarena Varela from FKIE.
As the scientists explain, the entire LUCY system ultimately functions in a similar way to our hearing ability: sound information is first recorded by a receiving unit - in our case the two ears - and passed on to the brain, where it is analyzed. In LUCY, the receiving unit comprising 48 microphones is connected to a signal processing module that is used to localize the sound sources. “Spatial hearing naturally works better with 48 or more microphones than with two acoustic sensors, and targeted listening in one direction also works better, as does listening away,” says research group leader Marc Oispuu. The system also detects frequencies that the human ear cannot register, the developers emphasize.
AI recognizes acoustic distress signals
But what about the loud whirring of the drone and other disturbing noises? The researchers explain that they can be effectively hidden by the signal processing unit. Adaptive filters and artificial intelligence (AI) are used. The AI is also specialized in recognizing sound patterns such as screaming, hitting or clapping that people typically use in emergency situations. The AI system was trained to distinguish between relevant and unimportant noises using a noise database, the developers explain.
A drone equipped with LUCY can detect the finest noises, precisely determine their angle of incidence and also classify them. Because the system is small, inexpensive and lightweight, it could be used on a larger scale in disaster response, the developers say. The received location data can then be transmitted to the rescue teams. The exact positions of those seeking help in the disaster region appear on tablets. The developers are now working on making the system available for use soon. Further optimizations are also planned: Among other things, the 48 microphones are to be expanded to 256 in the future in order to further improve LUCY's directional clairaudience.
Source: Fraunhofer Society