The power of sound: Exploring the auditory influence on visual search efficiency.

Abstract

In a dynamic visual search environment, a synchronous and meaningless auditory signal (pip) that corresponds with a change in a visual target promotes the efficiency of visual search (pop out), which is known as the pip-and-pop effect. We conducted three experiments to investigate the mechanism of the pip-and-pop effect. Using the eye movement technique, we manipulated the interval rhythm (Exp. 1) and interval duration time (Exp. 2) of dynamic color changes in visual stimuli in the dynamic visual search paradigm to ensure that there was a significant pip-and-pop effect. In Exp. 3, we modulated the appearance of the sound by employing a visual-only condition, an auditory target condition (synchronized sounds), an auditory oddball condition (a high-frequency sound in a series of low-frequency sounds), an omitted oddball condition (an omitted sound in a series of sounds) and an auditory non-oddball condition (the last of the four sounds). We aim to clarify the role of audiovisual cross-modal information in the pip-and-pop effect by comparing different conditions. The search time results showed that a significant pip-and-pop effect was found for the auditory target, auditory oddball and auditory non-oddball conditions. The eye movement results revealed an increase in the fixation duration and a decrease in the number of fixations for the auditory target and auditory oddball conditions. Our findings suggest that the pip-and-pop effect is indeed a cross-modal effect. Furthermore, the interaction between auditory and visual information is necessary for the pip-and-pop effect, whereas auditory oddball stimuli attract attention and therefore moderate this effect. Our study provides a solution for the pip-and-pop effect mechanism in a dynamic visual search paradigm.