Recognizing faces is crucial to our social lives. Several regions in our brain are therefore specialized for this. But what about people who are blind from birth? For a study, researchers have now equipped blind people with a device that converts images into sounds, enabling them to recognize pictograms of faces, houses and geometric shapes based on their acoustic features. Brain scans revealed that faces perceived in this way are processed in the same regions that are responsible for facial recognition in sighted people.
Different regions in our brain are involved in our ability to memorize faces and recognize familiar people - a fundamental skill for our social life. But is this ability innate to us, or does it depend on early childhood visual experiences with faces? In search of the answer to this question, people who have never seen a face in their life, since they were born blind, can help.
Pixels converted into sounds
A team led by Paula Plaza from Georgetown University in Washington DC has now investigated to what extent brain areas for face recognition are also activated in blind people when they perceive information about faces through other sensory channels. “It has been known for some time that blind people can compensate for the loss of their vision to a certain extent by using their other senses,” says Plaza's colleague Josef Rauschecker. “In our study, we examined the extent to which this plasticity exists between seeing and hearing.”
To do this, the team used a so-called sensory substitution device. This records the environment with a camera and converts the visual information into sounds. These sounds are played almost in real time via headphones, with the pitch and the stereo sound indicating where the corresponding pixel is located. A point at the top left of the field of vision becomes a high tone played in the left ear, a point at the bottom right becomes a low tone in the right ear. A vertical line in the middle of the field of vision is emitted as a combination of high and low tones in both ears. Little by little, the overall picture can be grasped.
“See” with your ears
For the study, six people who were either blind from birth or who lost their vision completely within their first two years of life trained to recognize simple shapes and pictograms using this device. As a control group, ten sighted people who were blindfolded during training and experiments completed the same training. After ten one-hour training sessions, both the blind and the sighted test subjects were able to recognize various pictograms based solely on the tones played with an accuracy of 85 percent. These included happy, sad and neutral faces, houses of different widths and heights, and different geometric shapes.
For the actual experiment, the researchers again played the acoustic patterns for the corresponding pictograms to their test subjects and recorded their brain activity using functional magnetic resonance imaging (fMRI). In order to ensure that the test subjects did not simply memorize tone sequences, but actually decoded the respective patterns, they played the tones in different orders - for example, in one run first the signals for the pixels on the top left, in another run first the signals for the pixels at the top right.
Facial area in the brain activated
As in the training runs, blind and sighted test subjects achieved similarly high scores when recognizing the pictograms. In order to find out to what extent the stylized faces play a special role compared to the other motifs, the researchers paid particular attention to the so-called fusiform facial area in the temporal lobe of the brain - an important region for processing faces that normally reacts to visual stimuli.
And indeed: In all test subjects, the fusiform facial area in the brain was more active when they perceived faces instead of houses or other shapes - even though the corresponding signals were only conveyed acoustically. "Our study shows that the fusiform facial area encodes the 'concept' of a face regardless of the input channel or visual experience," explains Plaza. "This suggests that the development of the fusiform face recognition area does not depend on experience with visual faces, but rather on experience with the geometry of facial configurations, which can also be mediated by other sensory channels."
Starting point for further developments
It was striking that the activation of the brain by sound took place primarily in the left fusiform area of the face in the blind subjects, whereas in the sighted test subjects it occurred primarily in the right fusiform area of the face. "We think that the left-right difference between blind and non-blind people may have to do with how the left and right sides of the fusiform area process faces - either as connected patterns or as separate parts," says Rauschecker.
This could be an important clue for the researchers to further develop their sensory substitution device. So far, the resolution is so low that only simple pictograms can be displayed. However, further information about processing in the brain could help to convey even more complex images in an understandable way. “We would like to find out whether it is possible for blind people to learn to recognize people based on acoustically transmitted images,” says Rauschecker. “This may require a lot more practice with our device, but now that we have pinpointed the region of the brain where translation occurs, we may be able to fine-tune our processes better.”
Source: Paula Plaza (Georgetown University Medical Center, Washington, DC, USA) et al., PLoS ONE, doi: 10.1371/journal.pone.0286512