Distinct portions of superior temporal sulcus combine auditory representations with different visual streams.

Abstract

In humans, the superior temporal sulcus (STS) combines auditory and visual information. However, the extent to which it relies on visual information from the ventral or dorsal stream remains uncertain. To address this, we analyzed open-source functional magnetic resonance imaging data collected from 15 participants (6 females and 9 males) as they watched a movie. We used artificial neural networks to investigate the relationship between multivariate response patterns in auditory cortex, the two visual streams, and the rest of the brain, finding that distinct portions of the STS combine information from the two visual streams with auditory information.Significance Statement The STS combines auditory and visual inputs. However, visual information is processed along a ventral and a dorsal stream, and the extent to which these streams contribute to the combination of audio-visual information is poorly understood. Is auditory information combined with visual information from both streams in a single centralized hub? Or do separate regions combine auditory information with ventral visual regions on one hand, and with dorsal visual regions on the other? To address this question, we employed a multivariate connectivity method based on artificial neural networks. Our findings reveal that information from the two visual streams is combined with auditory information in distinct portions of STS, offering new insights into the neural architecture underlying multisensory perception.