{"title":"It Sounds Cool: Exploring Sonification of Mid-Air Haptic Textures Exploration on Texture Judgments, Body Perception, and Motor Behaviour","authors":"Roberto Montano-Murillo;Dario Pittera;William Frier;Orestis Georgiou;Marianna Obrist;Patricia Cornelio","doi":"10.1109/TOH.2023.3320492","DOIUrl":null,"url":null,"abstract":"Ultrasonic mid-air haptic technology allows for the perceptual rendering of textured surfaces onto the user's hand. Unlike real textured surfaces, however, mid-air haptic feedback lacks implicit multisensory cues needed to reliably infer a texture's attributes (e.g., its roughness). In this article, we combined mid-air haptic textures with congruent sound feedback to investigate how sonification could influence people's 1) explicit judgment of the texture attributes, 2) explicit sensations of their own hand, and 3) implicit motor behavior during haptic exploration. Our results showed that audio cues (presented solely or combined with haptics) influenced participants' judgment of the texture attributes (roughness, hardness, moisture and viscosity), produced some hand sensations (the feeling of having a hand smoother, softer, looser, more flexible, colder, wetter and more natural), and changed participants' speed (moving faster or slower) while exploring the texture. We then conducted a principal component analysis to better understand and visualize the found results and conclude with a short discussion on how audio-haptic associations can be used to create embodied experiences in emerging application scenarios in the metaverse.","PeriodicalId":13215,"journal":{"name":"IEEE Transactions on Haptics","volume":null,"pages":null},"PeriodicalIF":2.4000,"publicationDate":"2023-10-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10285051","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Haptics","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10285051/","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, CYBERNETICS","Score":null,"Total":0}
引用次数: 0
Abstract
Ultrasonic mid-air haptic technology allows for the perceptual rendering of textured surfaces onto the user's hand. Unlike real textured surfaces, however, mid-air haptic feedback lacks implicit multisensory cues needed to reliably infer a texture's attributes (e.g., its roughness). In this article, we combined mid-air haptic textures with congruent sound feedback to investigate how sonification could influence people's 1) explicit judgment of the texture attributes, 2) explicit sensations of their own hand, and 3) implicit motor behavior during haptic exploration. Our results showed that audio cues (presented solely or combined with haptics) influenced participants' judgment of the texture attributes (roughness, hardness, moisture and viscosity), produced some hand sensations (the feeling of having a hand smoother, softer, looser, more flexible, colder, wetter and more natural), and changed participants' speed (moving faster or slower) while exploring the texture. We then conducted a principal component analysis to better understand and visualize the found results and conclude with a short discussion on how audio-haptic associations can be used to create embodied experiences in emerging application scenarios in the metaverse.
期刊介绍:
IEEE Transactions on Haptics (ToH) is a scholarly archival journal that addresses the science, technology, and applications associated with information acquisition and object manipulation through touch. Haptic interactions relevant to this journal include all aspects of manual exploration and manipulation of objects by humans, machines and interactions between the two, performed in real, virtual, teleoperated or networked environments. Research areas of relevance to this publication include, but are not limited to, the following topics: Human haptic and multi-sensory perception and action, Aspects of motor control that explicitly pertain to human haptics, Haptic interactions via passive or active tools and machines, Devices that sense, enable, or create haptic interactions locally or at a distance, Haptic rendering and its association with graphic and auditory rendering in virtual reality, Algorithms, controls, and dynamics of haptic devices, users, and interactions between the two, Human-machine performance and safety with haptic feedback, Haptics in the context of human-computer interactions, Systems and networks using haptic devices and interactions, including multi-modal feedback, Application of the above, for example in areas such as education, rehabilitation, medicine, computer-aided design, skills training, computer games, driver controls, simulation, and visualization.