{"title":"Does Task-Irrelevant Brightness Modulation Affect Auditory Contrast Processing? Exploring the Interplay Between Temporal Synchrony and Stimulus Salience","authors":"H. Chow, Danielle Briggs, V. Ciaramitaro","doi":"10.1163/22134808-bja10102","DOIUrl":"https://doi.org/10.1163/22134808-bja10102","url":null,"abstract":"\u0000Stimulus factors such as timing, spatial location, and stimulus effectiveness affect whether and how information across the senses is integrated. Extending recent work highlighting interactions between stimulus factors, here we investigated the influence of visual information on auditory processing, complementing previous studies on the influence of auditory information on visual processing. We hypothesized that task-irrelevant and spatially non-informative visual information would enhance auditory contrast processing, when visual information was at an optimal salience level and changed synchronously with the sound. We asked human observers to indicate the location of an amplitude-modulated white-noise sound, while its loudness against a constant white-noise background varied across trials. To test for the influence of task-irrelevant visual information, we modulated screen brightness smoothly (Experiment 1) or transiently (Experiment 2) in phase or out of phase with the amplitude modulation of the target sound. In addition, to test for the interaction between temporal synchrony and stimulus salience, maximum brightness varied systematically across trials. Auditory contrast thresholds were compared across conditions. Results showed that task-irrelevant visual information did not alter auditory contrast thresholds regardless of the nature of modulation of brightness, contrary to our expectations. Nonetheless, task-irrelevant visual information modulated in phase with the target sound reduced auditory contrast thresholds if we accounted for individual differences in the optimal salience required for the largest multisensory effects. Our results are discussed in light of several stimulus factors that might be critical in modulating multisensory enhancement.","PeriodicalId":51298,"journal":{"name":"Multisensory Research","volume":" ","pages":""},"PeriodicalIF":1.6,"publicationDate":"2023-05-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45356022","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Linking Auditory-Induced Bouncing and Auditory-Induced Illusory Crescents: an Individual-Differences Approach","authors":"Hauke S. Meyerhoff, M. Stegemann, C. Frings","doi":"10.1163/22134808-bja10100","DOIUrl":"https://doi.org/10.1163/22134808-bja10100","url":null,"abstract":"When two disks move toward each other, overlap, and then move apart, the visual system can resolve the ambiguity either as two disks streaming past each other or two disks bouncing off each other. Presenting a brief beep at the moment of overlap has been observed to increase the proportion of reported bouncing impressions (i.e., auditory-induced bouncing) as well as to reduce the perceived overlap between the disks (leaving a larger uncovered crescent; auditory-induced illusory crescents). Previous research has speculated about the relationship between both variables, but no direct evidence has been reported yet. We present an individual-differences study in which our participants completed the bouncing/streaming task as well as the illusory crescent task on two consecutive days (to obtain test–retest reliabilities). We obtained acceptable to good reliabilities for the effect of the tone in both dependent measures. Most importantly, auditory-induced bouncing and auditory-induced illusory crescents were correlated in the moderate range suggesting that both illusions are related and share common underlying cognitions. Yet, moderate correlations also indicate that both measures partially capture distinct aspects of the object correspondence.","PeriodicalId":51298,"journal":{"name":"Multisensory Research","volume":" ","pages":""},"PeriodicalIF":1.6,"publicationDate":"2023-05-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48645705","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Colour–Touch Cross-Modal Correspondence and Its Impact on Single-Modal Judgement in Multimodal Perception","authors":"Tianyi Yuan, P. Rau, Jingyu Zhao, Jian Zheng","doi":"10.1163/22134808-bja10098","DOIUrl":"https://doi.org/10.1163/22134808-bja10098","url":null,"abstract":"\u0000This study explored the colour–touch cross-modal correspondence and its impact on colour–touch multisensory perception. Two laboratory experiments were conducted based on a pre-experiment. In the first experiment, participants chose the colour according to their tactile sense against the vibration generated by the smartphone simulator. A positive cross-modal correspondence was obtained between the vibration amplitude and the colour chroma, indicating the existence of a colour–touch correspondence. The correspondence between vibration parameters and colour hue was preliminarily explored. The impact of the correspondence on the multisensory senses was investigated in the second experiment. Different stimulus combinations were played on a smartphone. The participants were asked to choose the perceived stimulus level. The data illustrated that a better correspondence improved judgement accuracy and reduced reactiontime. Overall, this study offers evidence for colour–touch cross-modal correspondence and presents its potential influence on the single-modal judgement in multimodal perception. The results can be applied in further cross-modal studies, assisting barrier-free society construction and multimodal interface design.","PeriodicalId":51298,"journal":{"name":"Multisensory Research","volume":" ","pages":""},"PeriodicalIF":1.6,"publicationDate":"2023-05-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44695454","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Audio–Visual Cross-Modal Correspondences of Perceived Urgency: Examination through a Speeded Discrimination Task","authors":"Kiichi Naka, K. Yamauchi","doi":"10.1163/22134808-bja10099","DOIUrl":"https://doi.org/10.1163/22134808-bja10099","url":null,"abstract":"\u0000When presenting information in vehicle cockpits, it is essential to convey an appropriate urgency to the drivers. Perceived urgency has been investigated over the years for each modality, particularly audition and vision. However, the interaction between the modalities of perceived urgency has rarely been examined. To expand the insight into the design application of information presentation, we investigated the audio–visual interaction of perceived urgency using a priming task that involved speeded visual-target discrimination. A total of 60 auditory stimuli were created using a synthesizer. In addition, 13 color squares were used as visual stimuli. Three auditory stimuli with high, medium, and low perceived urgency, and two visual stimuli with high and low perceived urgency were selected based on a subjective evaluation test using a seven-point scale. A priming task was conducted to examine the cross-modal interaction of perceived urgency. Auditory stimuli were presented as prime stimuli, and the participants were asked to discriminate the visual target as quickly as possible. The results revealed that auditory stimuli with high and low perceived urgency facilitated responses to each visual stimulus with similar perceived urgency relative to each visual stimulus with different perceived urgency. The auditory stimulus with medium perceived urgency also facilitated responses to the visual stimulus with high perceived urgency relative to low. The present study shows that cross-modal correspondences can be observed when the stimuli are selected based on their subjective perceived urgency.","PeriodicalId":51298,"journal":{"name":"Multisensory Research","volume":"39 3","pages":""},"PeriodicalIF":1.6,"publicationDate":"2023-05-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41274780","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Association Between Body Tilt and Egocentric Estimates Near Upright.","authors":"Keisuke Tani, Shintaro Uehara, Satoshi Tanaka","doi":"10.1163/22134808-bja10097","DOIUrl":"https://doi.org/10.1163/22134808-bja10097","url":null,"abstract":"<p><p>The mechanisms underlying geocentric (orientations of an object or the body relative to 'gravity') and egocentric estimates (object orientation relative to the 'body') have each been examined; however, little is known regarding the association between these estimates, especially when the body is nearly upright. To address this, we conducted two psychophysical experiments. In Experiment 1, participants estimated the direction of a visual line (subjective visual vertical; SVV) and their own body relative to gravity (subjective body tilt; SBT) and the direction of a visual line relative to the body longitudinal axis (subjective visual body axis; SVBA) during a small-range whole-body roll tilt. We evaluated the correlations between performance on each of these tasks as covariates of actual body tilt angles. Our results showed a significant correlation of performance (estimation errors) on the SVBA task with performance on the SBT task but not performance on the SVV task at the group level after adjusting for the actual body tilt angles, suggesting a link between the estimates for SVBA and SBT tasks. To confirm this relationship, in Experiment 2, we further assessed whether manipulating the subjective direction of the body axis by providing visual feedback in the SVBA task subsequently affected SBT performance. We found that feedback in the SVBA task significantly shifted the SBT angles even when the actual body angles were identical. The observed association between SVBA and SBT performance supports at least a partially shared mechanism underlying body tilt and egocentric estimates when the body is nearly upright.</p>","PeriodicalId":51298,"journal":{"name":"Multisensory Research","volume":"36 4","pages":"367-386"},"PeriodicalIF":1.6,"publicationDate":"2023-04-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9421378","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Explaining Visual Shape-Taste Crossmodal Correspondences.","authors":"Charles Spence","doi":"10.1163/22134808-bja10096","DOIUrl":"https://doi.org/10.1163/22134808-bja10096","url":null,"abstract":"<p><p>A growing body of experimental research now demonstrates that neurologically normal individuals associate different taste qualities with design features such as curvature, symmetry, orientation, texture and movement. The form of everything from the food itself through to the curvature of the plateware on which it happens to be served, and from glassware to typeface, not to mention the shapes of/on food product packaging have all been shown to influence people's taste expectations, and, on occasion, also their taste/food experiences. Although the origins of shape-taste and other form-taste crossmodal correspondences have yet to be fully worked out, it would appear that shape qualities are occasionally elicited directly. However, more often, there may be a metaphorical attempt to translate the temporal qualities of taste sensations into a spatial analogue. At the same time, emotional mediation may sometimes also play a role in the affinity people experience between shape properties and taste. And finally, it should be acknowledged that associative learning of the relation between packaging shapes, glassware shapes, logos, labels and iconic food forms that commonly co-occur with specific taste properties (i.e., in the case of branded food products) may also play an important role in determining the nature of shape-taste correspondences. Ultimately, however, any attempt to use such shape-taste correspondences to nudge people's behaviour/perception in the real world is made challenging due to the fact that shape properties are associated with multiple qualities, and not just taste.</p>","PeriodicalId":51298,"journal":{"name":"Multisensory Research","volume":"36 4","pages":"313-345"},"PeriodicalIF":1.6,"publicationDate":"2023-03-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9790067","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Off-Vertical Body Orientation Delays the Perceived Onset of Visual Motion.","authors":"William Chung, Michael Barnett-Cowan","doi":"10.1163/22134808-bja10095","DOIUrl":"https://doi.org/10.1163/22134808-bja10095","url":null,"abstract":"<p><p>The integration of vestibular, visual and body cues is a fundamental process in the perception of self-motion and is commonly experienced in an upright posture. However, when the body is tilted in an off-vertical orientation these signals are no longer aligned relative to the influence of gravity. In this study, the perceived timing of visual motion was examined in the presence of sensory conflict introduced by manipulating the orientation of the body, generating a mismatch between body and vestibular cues due to gravity and creating an ambiguous vestibular signal of either head tilt or translation. In a series of temporal-order judgment tasks, participants reported the perceived onset of a visual scene simulating rotation around the yaw axis presented in virtual reality with a paired auditory tone while in an upright, supine and side-recumbent body position. The results revealed that the perceived onset of visual motion was further delayed from zero (i.e., true simultaneity between visual onset and a reference auditory tone) by approximately an additional 30 ms when viewed in a supine or side-recumbent orientation compared to an upright posture. There were also no significant differences in the timing estimates of the visual motion between all the non-upright orientations. This indicates that the perceived timing of visual motion is negatively impacted by the presence of conflict in the vestibular and body signals due to the direction of gravity and body orientation, even when the mismatch is not in the direct plane of the axis of rotation.</p>","PeriodicalId":51298,"journal":{"name":"Multisensory Research","volume":"36 4","pages":"347-366"},"PeriodicalIF":1.6,"publicationDate":"2023-03-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9790065","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Faith Kimmet, Samantha Pedersen, Victoria Cardenas, Camila Rubiera, Grey Johnson, Addison Sans, Matthew Baldwin, Brian Odegaard
{"title":"Metacognition and Causal Inference in Audiovisual Speech.","authors":"Faith Kimmet, Samantha Pedersen, Victoria Cardenas, Camila Rubiera, Grey Johnson, Addison Sans, Matthew Baldwin, Brian Odegaard","doi":"10.1163/22134808-bja10094","DOIUrl":"https://doi.org/10.1163/22134808-bja10094","url":null,"abstract":"<p><p>In multisensory environments, our brains perform causal inference to estimate which sources produce specific sensory signals. Decades of research have revealed the dynamics which underlie this process of causal inference for multisensory (audiovisual) signals, including how temporal, spatial, and semantic relationships between stimuli influence the brain's decision about whether to integrate or segregate. However, presently, very little is known about the relationship between metacognition and multisensory integration, and the characteristics of perceptual confidence for audiovisual signals. In this investigation, we ask two questions about the relationship between metacognition and multisensory causal inference: are observers' confidence ratings for judgments about Congruent, McGurk, and Rarely Integrated speech similar, or different? And do confidence judgments distinguish between these three scenarios when the perceived syllable is identical? To answer these questions, 92 online participants completed experiments where on each trial, participants reported which syllable they perceived, and rated confidence in their judgment. Results from Experiment 1 showed that confidence ratings were quite similar across Congruent speech, McGurk speech, and Rarely Integrated speech. In Experiment 2, when the perceived syllable for congruent and McGurk videos was matched, confidence scores were higher for congruent stimuli compared to McGurk stimuli. In Experiment 3, when the perceived syllable was matched between McGurk and Rarely Integrated stimuli, confidence judgments were similar between the two conditions. Together, these results provide evidence of the capacities and limitations of metacognition's ability to distinguish between different sources of multisensory information.</p>","PeriodicalId":51298,"journal":{"name":"Multisensory Research","volume":"36 3","pages":"289-311"},"PeriodicalIF":1.6,"publicationDate":"2023-02-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9790066","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Assessing the Effects of Exercise, Cognitive Demand, and Rest on Audiovisual Multisensory Processing in Older Adults: A Pilot Study.","authors":"Aysha Basharat, Michael Barnett-Cowan","doi":"10.1163/22134808-bja10085","DOIUrl":"https://doi.org/10.1163/22134808-bja10085","url":null,"abstract":"<p><p>A single bout of aerobic exercise is related to positive changes in higher-order cognitive function among older adults; however, the impact of aerobic exercise on multisensory processing remains unclear. Here we assessed the effects of a single bout of aerobic exercise on commonly utilized tasks that measure audiovisual multisensory processing: response time (RT), simultaneity judgements (SJ), and temporal-order judgements (TOJ), in a pilot study. To our knowledge this is the first effort to investigate the effects of three well-controlled intervention conditions on multisensory processing: resting, completing a cognitively demanding task, and performing aerobic exercise for 20 minutes. Our results indicate that the window of time within which stimuli from different modalities are integrated and perceived as simultaneous (temporal binding window; TBW) is malleable and changes after each intervention condition for both the SJ and TOJ tasks. Specifically, the TBW consistently became narrower post exercise while consistently increasing in width post rest, suggesting that aerobic exercise may improve temporal perception precision via broad neural change rather than targeting the specific networks that subserve either the SJ or TOJ tasks individually. The results from the RT task further support our findings of malleability of the multisensory processing system, as changes in performance, as assessed through cumulative probability models, were observed after each intervention condition. An increase in integration (i.e., greater magnitude of multisensory effect) however, was only found after a single bout of aerobic exercise. Overall, our results indicate that exercise uniquely affects the central nervous system and may broadly affect multisensory processing.</p>","PeriodicalId":51298,"journal":{"name":"Multisensory Research","volume":"36 3","pages":"213-262"},"PeriodicalIF":1.6,"publicationDate":"2023-01-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9382058","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Kacie Dunham, Alisa Zoltowski, Jacob I Feldman, Samona Davis, Baxter Rogers, Michelle D Failla, Mark T Wallace, Carissa J Cascio, Tiffany G Woynaroski
{"title":"Neural Correlates of Audiovisual Speech Processing in Autistic and Non-Autistic Youth.","authors":"Kacie Dunham, Alisa Zoltowski, Jacob I Feldman, Samona Davis, Baxter Rogers, Michelle D Failla, Mark T Wallace, Carissa J Cascio, Tiffany G Woynaroski","doi":"10.1163/22134808-bja10093","DOIUrl":"https://doi.org/10.1163/22134808-bja10093","url":null,"abstract":"<p><p>Autistic youth demonstrate differences in processing multisensory information, particularly in temporal processing of multisensory speech. Extensive research has identified several key brain regions for multisensory speech processing in non-autistic adults, including the superior temporal sulcus (STS) and insula, but it is unclear to what extent these regions are involved in temporal processing of multisensory speech in autistic youth. As a first step in exploring the neural substrates of multisensory temporal processing in this clinical population, we employed functional magnetic resonance imaging (fMRI) with a simultaneity-judgment audiovisual speech task. Eighteen autistic youth and a comparison group of 20 non-autistic youth matched on chronological age, biological sex, and gender participated. Results extend prior findings from studies of non-autistic adults, with non-autistic youth demonstrating responses in several similar regions as previously implicated in adult temporal processing of multisensory speech. Autistic youth demonstrated responses in fewer of the multisensory regions identified in adult studies; responses were limited to visual and motor cortices. Group responses in the middle temporal gyrus significantly interacted with age; younger autistic individuals showed reduced MTG responses whereas older individuals showed comparable MTG responses relative to non-autistic controls. Across groups, responses in the precuneus covaried with task accuracy, and anterior temporal and insula responses covaried with nonverbal IQ. These preliminary findings suggest possible differences in neural mechanisms of audiovisual processing in autistic youth while highlighting the need to consider participant characteristics in future, larger-scale studies exploring the neural basis of multisensory function in autism.</p>","PeriodicalId":51298,"journal":{"name":"Multisensory Research","volume":"36 3","pages":"263-288"},"PeriodicalIF":1.6,"publicationDate":"2023-01-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10121891/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9382061","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}