Assessing the limits on size-pitch mapping reveals the interplay between top-down and bottom-up influences on relative crossmodal correspondences.

Abstract

Certain sensory dimensions, such as visual size and auditory pitch, are consistently associated, resulting in performance facilitation or inhibition. The mechanisms underlying these crossmodal correspondences are still the subject of debate: The relative or absolute nature of crossmodal mappings is connected to this debate, as an absolute mapping points to a bottom-up process, whereas a relative one is evidence of stronger top-down influences. Three experiments were conducted (including overall N = 207 participants), based on two different tasks, designed to explore a wide range of size-pitch crossmodal mappings. In Experiment 1, the participants were instructed to freely manipulate stimuli varing along a given dimension to 'match' the other. The results revealed evidence for a quasi-absolute mapping, but the correspondences shifted depending on the participants' auditory or visual attentional focus. In Experiment 2, the participants performed a visual speeded categorization task, involving a wide range of auditory task-irrelevant pitches, including the "preferred" ones, estimated on the basis of the results of Experiment 1. The results revealed a rather relative mapping, corroborating a top-down influence on the correspondence effect. Experiment 3 was designed to determine whether the relative mapping involved has boundary. The results confirmed that the larger the interval between pitches (i.e., more perceptually salient), the stronger the congruence effect, thus highlighting bottom-up facilitation. Taken together, these findings reveal that the size-pitch correspondences are sensitive to task-related top-down factors, as well as to stimulus-related bottom-up influences, ultimately revealing the adaptive nature of this kind of multisensory integration.