‘screens in cars will disappear’

Augmented reality (AR) technology will make screens and buttons in cars superfluous in five to ten years. Mercedes-Benz development engineers are convinced of this. To prove that it really works, they have created a concept model of the car interior of the future. Mercedes’ latest study model, the Vision One-Eleven, served as the basis: a striking, futuristic car full of new design and technology ideas.

For this innovation, the car driver must put on glasses with augmented reality. Then, as it were, an artificial, digital layer with graphic images is superimposed over reality via the digital spectacle lens. So through the glasses you can still see everything that is actually in your car and what is happening outside the car.

As soon as you take a seat behind the wheel, you notice that buttons or screens are missing. If you then put on the special glasses, they will still appear. All kinds of buttons in and around the dashboard are conjured up, when in reality they are not there. Sometimes they only appear when you need them. For example, when a phone call comes in, you’ll see a button that lets you accept or decline the call. The control buttons are also interactive. They beep to confirm you’re touching them, and pointing at them can also make them extra large and even clearer.

Mercedes Benz

With the technology, you can also see, for example, the road map from the navigation system hanging above the center console as an apparently floating and transparent piece of paper. But the most spectacular thing about the partly digitized car interior is the view of the surroundings outside the car. You see the outside world lifelike and in detail. Actually, just like you normally look outside through the windows of the car. Only the study model is in a bare studio and the roads, traffic and mountain landscapes you see all come from the computer.

Herein lies the greatest challenge for the future of this technology. A lot of data is needed to be able to create such a realistic picture. And it must have been captured with advanced cameras at some point. Think of images such as Google, which has been capturing with Streetview for several years now. Another source is the digital HD road maps (with high resolution) that map makers such as Google and TomTom have been making for a few years now, for navigation with self-driving cars. In the innovation from Mercedes-Benz, all this is supplemented by an up-to-date display of everything around the car. From flower box to lamppost, the car and AR technology will soon have to see everything. Think of parked cars and cars, cyclists and pedestrians in motion.

This data does not come from images stored in the cloud, but from lidar, among other things: the laser radar that forms a high-precision image of everything it ‘sees’ every second, based on reflected laser beams. The latest cars, such as the Volvo EX90 and Nio ET7, already have such a lidar on board. The Mercedes-Benz innovation also uses the data reported by other cars driving in the vicinity: a traffic jam, an accident, a broken down car or a rain shower. Cameras and (ordinary) radars in the car help to complete the current image.

The fact that the driver sees artificial images of the driving environment brings numerous additional benefits. You have a much better view of the surroundings and the dangers on the road. And it is ideal when driving at night, in thick fog or during a heavy rain shower. In addition, the digitally sketched image is unaffected by window pillars or other body parts of the car that normally restrict the driver’s view. You look through everything, as it were, from the window pillars, to the engine compartment, the trunk and the doors of the car. It would even be possible to omit the windows from cars from now on.

Mercedes Benz

Sunglasses

How quickly the new technology can be applied in cars depends on how quickly the necessary data becomes available to create the images. Google’s camera cars are far from driving all over the world to capture everything. It is quite conceivable that, just as with the introduction of navigation systems, only a limited number of regions or countries will first be digitally mapped.

It remains to be seen to what extent the system is user-friendly. At the moment, many motorists will not want to get behind the wheel with thick, heavy computer glasses on their head. The AR glasses used by the Mercedes development department in Silicon Valley are the Magic Leap 2. They cost just under €4,000, but they are one of the most compact and light models currently available. Mercedes assumes that the AR glasses will be reduced to the size of normal sunglasses within a few years. And work is even underway to apply AR technology in contact lenses.

AR glasses are now mainly used by gamers. But that AR glasses have a great future, Apple underlined again this month with the unveiling of the Apple Vision Pro glasses. That was one of the most important new products during the annual product presentation. Those glasses also cost around 4000 euros and Apple has high expectations of them.

AR glasses are now being used not only by gamers, but also in medical science: some surgeons operate with them. Then the data from pre-made scans is made visible in such glasses. Because a surgeon has to operate with great precision, accuracy is very important. Thanks to the AR glasses, the surgeon can see, for example, exactly where the blood vessels run under the patient’s skin.

In the car, AR technology has another purpose. “We want to relieve the driver’s stress,” say the Mercedes-Benz development engineers. “The current generation of cars offers so many options due to the electronics that operating them quickly becomes complicated. Poor visibility in the dark or in bad weather also makes a car journey less relaxing. Less stress, more ease of operation and safer traffic are possible thanks to this technology.”