The Effect of Induced Emotional States on The Magnitude of Cross-Modal Correspondence Effect

Abstract

Cross-modal correspondence effect (i.e., facilitated processing of congruent stimuli from different modalities) occurs not only when simple multi-modal sensory stimuli are processed together, but also during their simultaneous processing with words with emotional and spatial connotations. We tested a hypothesis that the magnitude of cross-modal correspondence effect, arising from concurrent processing of basic sensory and verbal stimuli, is differentially modulated by individual’s emotional state. Thirty-six volunteers (26 females, 18–34 years old) watched videos that evoked positive, negative, or neutral emotional states. This was followed by the main task in which they were presented with sounds of different pitch (low: 1000 Hz; high: 2000 Hz) simultaneously with words that differed in their emotional valence and were associated with different parts of space (low/high). The participant’s task was to identify the pitch (low/high) of the non-verbal sound stimuli. Two-way mixed ANOVA and subsequent pairwise comparisons (Student's t-test for dependent samples) were used to compare both mean reaction times and estimated parameters of the ex-Gaussian distribution. The results showed that the audiovisual correspondence effect became manifested in faster responses to congruent stimulus combinations compared with non-congruent ones (t(35) = -3.20, p = .005, dz = -0.53, 95% CI [-0.89, -0.18]). However, we did not find a large size effect of the induced emotional state on the magnitude of this correspondence effect (F(4, 68) = 0.49, p = 0.744, = .001). This result may be explained either by robustness of cross-modal correspondence effect and its resilience to emotional influence or by specific limitations of present study design.