Spatiotemporal Congruency Modulates Weighting of Visuotactile Information in Displacement Judgments

Abstract

Combining or integrating information from multiple senses often provides richer and more reliable estimates for the perception of objects and events. In daily life, sensory information from the same source often is in close spatiotemporal proximity. This can be an important determinant of whether and how multisensory signals are combined. The introduction of advanced technical display systems allows to present multisensory information in virtual environments. However, technical displays can lack the spatiotemporal fidelity of the real world due the rendering delays. Thus, any spatiotemporal incongruency could alter how information is combined. In the current study we tested this by investigating if and how spatially and temporally discrepant tactile displacement cues can supplement imprecise visual displacement cues. Participants performed a visual displacement task with visual and tactile displacement cues under spatial and temporal incongruency conditions. We modelled how participants combined visual and tactile information in visuotactile condition using their performance in visual only condition. We found that temporal incongruency lead to an increase in tactile weights although they were correlated with the congruency condition. In contrast, the spatial incongruency led to individual differences altering cue combination strategies. Our results illustrate the importance of spatiotemporal congruency for combining tactile and visual cues when making visual displacement judgments. Given the altered cue combination strategies and individual differences, we recommend developers to adopt individual spatiotemporal calibration procedures to improve the efficiency of the sensory augmentation.