Reconstruction of speech spectrogram based on non-invasive EEG signal

Abstract

Decoding neural activity into speech could enable natural conversations for people who are unable to communicate as a result of neurological diseases. Studies have proven that speech could be directly recognized or synthesized from intracranial recordings. However, intracranial electrocorticography is invasive, thus not comfortable for patients. By the acoustic representation of speech in the high-level brain cortex, we successfully reconstructed a speech spectrogram from non-invasive electroencephalography (EEG), which has similar accuracy to previous intracranial recording. As well as the reported superior temporal gyrus, premotor cortex, and inferior frontal gyrus, we also found speech representations in several other cortices such as an entorhinal, fusiform, and temporal pole. The intelligibility of the recovered speech in this study was not high enough, however, our findings show a possibility to reconstruct speech from non-invasive EEG in the future.