Audiovisual reproduction in surrounding display: Effect of spatial width of audio and video

Abstract

Current perception-based quality metrics for unimodal systems cannot reflect the perceived quality in multimodal situations, and better understanding of multimodal perceptual mechanisms is needed. In this work, audiovisual perception was studied with an immersive audiovisual display. The aim was to observe cross-modal interaction of auditory and visual modalities, when the spatial width of audio and video reproduction were limited and overall perceived degradation evaluated. The results show both audio and video width affect the perceived degradation of a stimulus. The effect of audio width decreases as video width is decreased. Constrained correspondence analysis suggests the reasons for highest perceived degradation to be wrong audio direction, reduced video width and missing essential content.