Neural dynamics decoding of the influence of modal properties on referential shifting: A univariate and multivariate EEG analysis

Abstract

In linguistics, the referential target of pronouns can shift depending on context, known as “shifting.” Previous research has mainly focused on how contextual factors affect this shift, without exploring whether different perceptual modalities (e.g., visual and auditory) also influence shifting. To address this, the current study used an oddball paradigm to examine the neural dynamics of different pronouns across perceptual modalities. Results showed that in the visual modality, first-person and third-person pronouns differed significantly in neural activity during the N400 window, indicating self-other separation. MVPA results confirmed this separation in ERP and theta signals. In the auditory modality, second-person and third-person pronouns also exhibited significant separation in the N400 window, similar to the visual pattern. In the alpha band, significant separation between second-person and third-person pronouns was observed only in the auditory modality. These findings suggest that shifting effects are influenced by perceptual modalities in addition to context.