Visuo-Haptic Rendering of the Hand during 3D Manipulation in Augmented Reality

IF 2.4 3区计算机科学 Q2 COMPUTER SCIENCE, CYBERNETICS

IEEE Transactions on Haptics Pub Date : 2024-01-26 DOI:10.1109/TOH.2024.3358910

Erwan Normand;Claudio Pacchierotti;Eric Marchand;Maud Marchal

{"title":"Visuo-Haptic Rendering of the Hand during 3D Manipulation in Augmented Reality","authors":"Erwan Normand;Claudio Pacchierotti;Eric Marchand;Maud Marchal","doi":"10.1109/TOH.2024.3358910","DOIUrl":null,"url":null,"abstract":"Manipulating virtual objects with bare hands is a key interaction in Augmented Reality (AR) applications. However, there are still several limitations that affect the manipulation, including the lack of mutual visual occlusion between virtual and real content as well as the lack of haptic sensations. To address the two abovementioned matters, the role of the visuo-haptic rendering of the hand as sensory feedback is investigated. The first experiment explores the effect of showing the hand of the user as seen by the AR system through an avatar, comparing six visual hand rendering. The second experiment explores the effect of the visuo-haptic hand rendering by comparing two vibrotactile contact techniques provided at four delocalized positions on the hand and combined with the two most representative visual hand renderings from the first experiment. Results show that delocalized vibrotactile haptic hand rendering improved perceived effectiveness, realism, and usefulness when provided close to the contact point. However, the farthest rendering position, i.e., on the contralateral hand, gave the best performance even though it was largely disliked. The visual hand rendering was perceived as less necessary for manipulation when the haptic hand rendering was available, but still provided useful feedback on the hand tracking.","PeriodicalId":13215,"journal":{"name":"IEEE Transactions on Haptics","volume":"17 2","pages":"277-291"},"PeriodicalIF":2.4000,"publicationDate":"2024-01-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Haptics","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10415248/","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, CYBERNETICS","Score":null,"Total":0}

引用次数: 0

Abstract

Manipulating virtual objects with bare hands is a key interaction in Augmented Reality (AR) applications. However, there are still several limitations that affect the manipulation, including the lack of mutual visual occlusion between virtual and real content as well as the lack of haptic sensations. To address the two abovementioned matters, the role of the visuo-haptic rendering of the hand as sensory feedback is investigated. The first experiment explores the effect of showing the hand of the user as seen by the AR system through an avatar, comparing six visual hand rendering. The second experiment explores the effect of the visuo-haptic hand rendering by comparing two vibrotactile contact techniques provided at four delocalized positions on the hand and combined with the two most representative visual hand renderings from the first experiment. Results show that delocalized vibrotactile haptic hand rendering improved perceived effectiveness, realism, and usefulness when provided close to the contact point. However, the farthest rendering position, i.e., on the contralateral hand, gave the best performance even though it was largely disliked. The visual hand rendering was perceived as less necessary for manipulation when the haptic hand rendering was available, but still provided useful feedback on the hand tracking.

查看原文本刊更多论文

在增强现实中进行 3D 操作时的手部视觉-触觉渲染。

徒手操控虚拟物体是增强现实（AR）应用中的一种关键交互方式。然而，在操作过程中仍存在一些限制因素，包括虚拟内容和现实内容之间缺乏相互视觉遮挡以及缺乏触觉。为了解决上述两个问题，我们研究了手的视觉-触觉渲染作为感觉反馈的作用。第一个实验探索了 AR 系统通过头像显示用户手部的效果，并对六种视觉手部渲染进行了比较。第二个实验通过比较在手的四个局部位置提供的两种振动触觉接触技术，并结合第一个实验中最具代表性的两种视觉手部效果图，来探索视觉触觉手部效果图的效果。结果表明，在靠近接触点的位置提供的局部振动触觉手部渲染提高了感知效果、真实性和实用性。然而，最远的渲染位置（即在对侧手掌上）的表现最好，尽管它在很大程度上不受欢迎。在有触觉手部渲染的情况下，视觉手部渲染被认为对操作的必要性较低，但仍能提供有用的手部跟踪反馈。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

IEEE Transactions on Haptics COMPUTER SCIENCE, CYBERNETICS-

CiteScore

5.90

自引率

13.80%

发文量

109

审稿时长

>12 weeks

期刊介绍： IEEE Transactions on Haptics (ToH) is a scholarly archival journal that addresses the science, technology, and applications associated with information acquisition and object manipulation through touch. Haptic interactions relevant to this journal include all aspects of manual exploration and manipulation of objects by humans, machines and interactions between the two, performed in real, virtual, teleoperated or networked environments. Research areas of relevance to this publication include, but are not limited to, the following topics: Human haptic and multi-sensory perception and action, Aspects of motor control that explicitly pertain to human haptics, Haptic interactions via passive or active tools and machines, Devices that sense, enable, or create haptic interactions locally or at a distance, Haptic rendering and its association with graphic and auditory rendering in virtual reality, Algorithms, controls, and dynamics of haptic devices, users, and interactions between the two, Human-machine performance and safety with haptic feedback, Haptics in the context of human-computer interactions, Systems and networks using haptic devices and interactions, including multi-modal feedback, Application of the above, for example in areas such as education, rehabilitation, medicine, computer-aided design, skills training, computer games, driver controls, simulation, and visualization.