Brain-computer interface creates text by decoding brain signals associated with handwriting

Brain-computer interface creates text by decoding brain signals associated with handwriting

Researchers for the first time, used an implanted sensor to record the brain signals associated with handwriting, and used those signals to create text on a computer in real time.

In a study published in the journal Nature, a clinical trial participant with cervical spinal cord injury used the system to "type" words on a computer at a rate of 90 characters per minute, more than double the previous record for typing with a brain-computer interface. This was done by the participant merely thinking about the hand motions involved in creating written letters.

The research team is hopeful that such a system could one day help to restore people's ability to communicate following paralysis caused by injury or illness.

For this latest study, the team wanted to find out if asking a participant to think about motions involved in writing letters and words by hand would be faster.

"We want to find new ways of letting people communicate faster," the author said. "This new system uses both the rich neural activity recorded by intracortical electrodes and the power of language models that, when applied to the neurally decoded letters, can create rapid and accurate text."

The trial participant, a 65-year-old (at the time of the study) man, was paralyzed from the neck down by a spinal cord injury. As part of the clinical trial, the authors placed two tiny electrodes about the size of a baby aspirin in a part of his brain associated with the movement of his right arm and hand. Using signals the sensors picked up from individual neurons when the man imagined writing, a machine learning algorithm recognized the patterns his brain produced with each letter. With this system, the man could copy sentences and answer questions at a rate similar to that of someone the same age typing on a smartphone.

The system is so fast because each letter elicits a highly distinctive activity pattern, making it relatively easy for the algorithm to distinguish one from another, the author says.

In 2012, the team published landmark research in which clinical trial participants were able, for the first time, to operate multidimensional robotic prosthetics using a BCI. That work has been followed by a steady stream of refinements to the system, as well as new clinical breakthroughs that have enabled people to directly control tablet apps and even move their own paralyzed limbs. Most recently, the team demonstrated the first human use of a wireless intracortical BCI that can transmit neural data at full bandwidth.