Investigating the temporal dynamics of auditory cortical activation to silent lipreading

Abstract

Neuroimaging research has demonstrated that observing visual speech in the absence of auditory speech activates primary auditory cortex. However, it remains unclear what this activation precisely reflects. It is well established that, during continuous auditory speech, neural activity in auditory cortex tracks the temporal envelope of the speech signal. Recently, it has been suggested that this process may in fact reflect an internal synthesis of the speech stream rather than the encoding of the envelope per se. Could silent lipreading therefore elicit a similar “entrainment” to the envelope in the absence of auditory speech? Here, we test this hypothesis by examining the impact of lipreading accuracy on envelope tracking using electroencephalography (EEG). We provide evidence to suggest that the EEG response over left temporal scalp tracks the unheard speech more faithfully during accurate lipreading. We also demonstrate that the envelope can be reconstructed from EEG data recorded during silent lipreading with accuracy above chance level. This could have implications for brain-computer interface technology.