Multisensory integration of a sound with stereo 3-D visual events

Abstract

The stream/bounce effect is an example of audio/visual interaction in which two identical luminance-defined targets in a 2-D display move toward one another from opposite sides of a display, coincide, and continue past one another along collinear trajectories. The targets can be perceived to either stream past or bounce off of one another. Streaming is the dominant perception in visual only displays while bouncing predominates when an auditory transient tone is presented at the point of coincidence. We extended previous findings on audio/visual interactions, using 3-D displays, and found the following two points. First, the sound-induced bias towards bouncing persists in spite of the introduction of spatial offsets in depth between the trajectories, which reduce the probability of motion reversals. Second, audio/visual interactions are similar for luminance-defined and disparity-defined displays, indicating that audio/visual interaction occurs at or beyond the visual processing stage where disparity-defined form is recovered.