Russian scientists have created a speech decoding model based on the analysis of brain activity. This was reported by the HSE’s press service.
Millions of people worldwide suffer from complete or partial speech impairment. They can be caused by both congenital diseases and brain injuries and strokes. In some cases, a device for articulation reading (“lip reading”) helps mute people, but paralyzed people do not have this ability.
Scientists from the National Research University Higher School of Economics and MGMSU. Evdokimov created a system for reading “mental speech”. It is based on a machine learning algorithm and is characterized by low invasiveness – it does not require a large number of electrodes in the brain. 5 to 9 electrodes with a different number of contacts were placed on the experimental subjects, and part of the skull was not removed, but holes were drilled.
To train the neural network in the experiment, the subject read 6 sentences aloud, each presented 30 to 60 times interspersed with the rest. The sentences had different linguistic structures and contained consonant words (for example, “Shura walks wide in wide trousers”). All sentences had 26 words. While reading the sentences, the electrodes recorded the electrical activity of the subject’s brain.
Then the algorithm was trained according to these data. As a result, the trained neural network was able to predict words with 55% accuracy for the first patient and 70% for the second patient based on brain activity signals recorded by a single 6-contact sEEG stick. On data from a single 8-contact ECoG strip.
The authors hope that their invention will significantly improve the quality of life of people with speech disorders.