With a new technology, researchers have made it possible for a paralyzed patient to write with his or her thoughts – just as quickly as non-paralyzed peers type on a smartphone. The test person imagined he was writing letters by hand. Electrodes in his brain passed the signals on to software that used them to generate text on a screen in real time. The approach could help more paralyzed, speech-impaired patients regain their communication skills in the future.
Strokes in the area of ​​the brain stem, diseases such as amyotrophic lateral sclerosis (ALS) and injuries to the cervical spine can result in those affected being fully conscious but unable to move or express themselves verbally. Interfaces between the brain and computer can help such people to regain contact with the outside world. For some technologies, electrodes glued to the scalp are sufficient; other, more precise processes use electrodes implanted in the brain.
Electrodes in the brain
A team led by Francis Willett from Standford University has now developed a new technology that makes it possible to convert thoughts into written text faster than ever before. While earlier systems rely on a patient imagining clicking letters on a screen, the new software evaluates the signals that arise when the patient imagines writing by hand. “We found that complex movements that involve changing speeds and curved paths such as handwriting can be interpreted more easily and quickly by our artificial intelligence algorithms than simpler movements, such as moving a cursor in a straight path at a steady speed,” explains Willett. “The letters of the alphabet are so different from each other that the signals are easier for the computer to distinguish.”
The subject in the current study was a man who was 65 years old at the time of the experiments and who has been paralyzed from the neck down since a spinal cord injury in 2007. For an earlier study, he had two small chips, each with 100 electrodes, implanted in the brain that record signals from neurons in the motor cortex – precisely from the region that controls hand movements. Willett and colleagues now also used these electrodes in his brain for their new technology.
New speed record
First, the test person trained the software by imagining several times in a row for given letters that he would write them by hand on an imaginary notepad. In this way, the artificial intelligence learned which brain signals belong to which letter. For the actual experiment, the researchers asked the test subjects to write down several sentences that the software had previously unknown. He succeeded in doing this with astonishing speed: he managed an average of 90 characters per minute – as many as healthy peers on a smartphone. This also topped the previous speed record in writing with the brain, which he himself had set in 2017 with the previous technology. Back then he had typed 40 characters per minute by imagining himself navigating the letters on a screen.
The test subject was also significantly faster with free writing than with previous technologies. The error rate was around one incorrectly recognized letter per 18 or 19 characters when copying sentences and one error per 11 or 12 characters when writing freely. To compensate for this, the researchers built in an auto-correction function – similar to that used in smartphones. This reduced the error rate to less than 1 percent when copying and a little over 2 percent when writing freely – very low values ​​compared to previous brain-computer interfaces.
Intuitive communication for paralyzed people
The results are particularly remarkable in view of the fact that the test subject has not written by hand for more than ten years due to his paralysis. “This study shows that the brain retains its ability to describe subtle movements – a decade after the body lost its ability to perform those movements,” says Willett.
In the future, the researchers want to test the technology with other patients who have lost their ability to speak or move due to illness or injury. “A major goal of our research is to restore fast, intuitive communication for people with severe speech or motor impairments,” says co-author Leigh Hochberg of Brown University in Rhode Island. “The demonstration of fast, precise neural decoding of handwriting marks an exciting new chapter in the development of clinically useful neurotechnologies.”
Source: Francis Willett (Stanford University) et al., Nature, doi: 10.1038 / s41586-021-03506-2