Usually, robots learn to move through their programming or in simulations. But a newly developed robot arm can instead observe itself via the camera and thus develop a kind of awareness of its own form and movement. This can also adapt to new situations and even compensate for damage, for example when it is bent. This could open up new opportunities for robots in household and industry.
When a toddler tries to reach for a toy for the first time, it watches his own hand. Through repeated experience, it learns to adapt the muscle movements to the desired result. Even as adults, we are dependent on this type of visual feedback when we learn complicated movements such as dancing. But while it is a matter of course for us to be able to observe ourselves, robots usually have to get by without this sensory impression. Instead, they are usually designed, tested and trained in simulations before they are allowed to move in reality for the first time.
Self -observation by camera
But now a team around Yuhang Hu from Columbia University in New York has equipped a robot arm with the ability to observe self -observation. The robot captures a picture of itself through cameras. “Our goal is a robot that understands his own body, adapts to damage and learns new skills without constant human programming,” explains HU. “Similar to people learn to dance by looking at their reflection, the robot uses video recordings to develop a kinematic self -confidence.”
Based on the two -dimensional camera images, the robot creates three -dimensional simulations of its own movements with the help of artificial intelligence. As a result, it is no longer necessary to train the robot with sophisticated virtual simulations before it is commissioned. “This ability not only saves the technical effort, but also enables the simulation to continue with the robot and develop with it when he uses, damages and adapts,” says Hus colleague Hod Lipson.
(Video: Jane Nisselson/Creative Machines Lab/Columbia Engineering)
Adaptation to damage
In fact, thanks to its newly obtained “self -awareness”, the robot is able to react to changes to his body. The researchers bent the robot arm, and his internal control also adapted to the new conditions independently. If, for example, the arm now moves around obstacles, the robot noticed its new shape and successfully performed the task despite the damage.
From the researcher’s point of view, this property could be useful for household robots, for example. “For example, imagine a vacuum cleaner robot or a personal assistant who realizes that his arm is bent after he has joined a piece of furniture,” says Hu. “Instead of having to fall out or repaired, he observes himself, adapts his movements and continues to work. This could make home robots more reliable – no constant reprogramming is required. ” Such adaptable robots could also reduce downtimes in industry and make production more resistant.
However, the robot now developed is still a long way from human self -perception. “We humans are intuitively aware of our body; We can move into the future and visualize the consequences of our actions long before we perform these actions in reality, ”explains Lipson. “Ultimately, we would like to give robots a similar ability to self -image.”
Source: Yuhang Hu (Columbia University, New York) et al., Nature Machine Intelligence, DOI: 10.1038/S42256-025-01006-W