Impaired neural encoding of naturalistic audiovisual speech in autism

Abstract

Visual cues from a speaker’s face can significantly improve speech comprehension in noisy environments through multisensory integration (MSI)—the process by which the brain combines auditory and visual inputs. Individuals with Autism Spectrum Disorder (ASD), however, often show atypical MSI, particularly during speech processing, which may contribute to the social communication difficulties central to the diagnosis. Understanding the neural basis of impaired MSI in ASD, especially during naturalistic speech, is critical for developing targeted interventions. Most neurophysiological studies have relied on simplified speech stimuli (e.g., isolated syllables or words), limiting their ecological validity. In this study, we used high-density EEG and linear encoding and decoding models to assess the neural processing of continuous audiovisual speech in adolescents and young adults with ASD (N = 23) and age-matched typically developing controls (N = 19). Participants watched and listened to naturalistic speech under auditory-only, visual-only, and audiovisual conditions, with varying levels of background noise, and were tasked with detecting a target word. Linear models were used to quantify cortical tracking of the speech envelope and phonetic features. In the audiovisual condition, the ASD group showed reduced behavioral performance and weaker neural tracking of both acoustic and phonetic features, relative to controls. In contrast, in the auditory-only condition, increasing background noise reduced behavioral and model performance similarly across groups. These results provide, for the first time, converging behavioral and neurophysiological evidence of impaired multisensory enhancement for continuous, natural speech in ASD.

Significance Statement

In adverse hearing conditions, seeing a speaker's face and their facial movements enhances speech comprehension through a process called multisensory integration, where the brain combines visual and auditory inputs to facilitate perception and communication. However, individuals with Autism Spectrum Disorder (ASD) often struggle with this process, particularly during speech comprehension. Previous findings using simple, discrete stimuli do not fully explain how the processing of continuous natural multisensory speech is affected in ASD. In our study, we used natural, continuous speech stimuli to compare the neural processing of various speech features in individuals with ASD and typically developing (TD) controls, across auditory and audiovisual conditions with varying levels of background noise. Our findings showed no group differences in the encoding of auditory-alone speech, with both groups similarly affected by increasing levels of noise. However, for audiovisual speech, individuals with ASD displayed reduced neural encoding of both the acoustic envelope and the phonetic features, marking neural processing impairment of continuous audiovisual multisensory speech in autism.