Mixed-reality for quadruped-robotic guidance in SAR tasks

IF 4.8 2区 工程技术 Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS
Christyan Cruz Ulloa, J. Cerro, A. Barrientos
{"title":"Mixed-reality for quadruped-robotic guidance in SAR tasks","authors":"Christyan Cruz Ulloa, J. Cerro, A. Barrientos","doi":"10.1093/jcde/qwad061","DOIUrl":null,"url":null,"abstract":"\n In recent years, exploration tasks in disaster environments, victim localization and primary assistance have been the main focuses of Search and Rescue (SAR) Robotics. Developing new technologies in Mixed Reality (M-R) and legged robotics has taken a big step in developing robust field applications in the Robotics field. This article presents MR-RAS (Mixed-Reality for Robotic Assistance), which aims to assist rescuers and protect their integrity when exploring post-disaster areas (against collapse, electrical, and toxic risks) by facilitating the robot’s gesture guidance and allowing them to manage interest visual information of the environment. Thus, ARTU-R (A1 Rescue Tasks UPM Robot) quadruped robot has been equipped with a sensory system (lidar, thermal and RGB-D cameras) to validate this proof of concept. On the other hand, Human-Robot interaction is executed by using the Hololens glasses. This work’s main contribution is the implementation and evaluation of a Mixed-Reality system based on a ROS-Unity solution, capable of managing at a high level the guidance of a complex legged robot through different interest zones (defined by a Neural Network and a vision system) of a post-disaster environment. The robot’s main tasks at each point visited involve detecting victims through thermal, RGB imaging and neural networks and assisting victims with medical equipment. Tests have been carried out in scenarios that recreate the conditions of post-disaster environments (debris, simulation of victims, etc.). An average efficiency improvement of 48% has been obtained when using the immersive interface and a time optimization of 21.4% compared to conventional interfaces. The proposed method has proven to improve rescuers’ immersive experience of controlling a complex robotic system.","PeriodicalId":48611,"journal":{"name":"Journal of Computational Design and Engineering","volume":"34 1","pages":"1479-1489"},"PeriodicalIF":4.8000,"publicationDate":"2023-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Computational Design and Engineering","FirstCategoryId":"5","ListUrlMain":"https://doi.org/10.1093/jcde/qwad061","RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS","Score":null,"Total":0}
引用次数: 1

Abstract

In recent years, exploration tasks in disaster environments, victim localization and primary assistance have been the main focuses of Search and Rescue (SAR) Robotics. Developing new technologies in Mixed Reality (M-R) and legged robotics has taken a big step in developing robust field applications in the Robotics field. This article presents MR-RAS (Mixed-Reality for Robotic Assistance), which aims to assist rescuers and protect their integrity when exploring post-disaster areas (against collapse, electrical, and toxic risks) by facilitating the robot’s gesture guidance and allowing them to manage interest visual information of the environment. Thus, ARTU-R (A1 Rescue Tasks UPM Robot) quadruped robot has been equipped with a sensory system (lidar, thermal and RGB-D cameras) to validate this proof of concept. On the other hand, Human-Robot interaction is executed by using the Hololens glasses. This work’s main contribution is the implementation and evaluation of a Mixed-Reality system based on a ROS-Unity solution, capable of managing at a high level the guidance of a complex legged robot through different interest zones (defined by a Neural Network and a vision system) of a post-disaster environment. The robot’s main tasks at each point visited involve detecting victims through thermal, RGB imaging and neural networks and assisting victims with medical equipment. Tests have been carried out in scenarios that recreate the conditions of post-disaster environments (debris, simulation of victims, etc.). An average efficiency improvement of 48% has been obtained when using the immersive interface and a time optimization of 21.4% compared to conventional interfaces. The proposed method has proven to improve rescuers’ immersive experience of controlling a complex robotic system.
SAR任务中四足机器人制导的混合现实
近年来,灾害环境下的探测任务、受害者定位和初步救援是搜救机器人的主要研究方向。混合现实(M-R)和腿式机器人的新技术的发展在机器人领域发展强大的现场应用方面迈出了一大步。本文介绍了MR-RAS(混合现实机器人援助),旨在通过促进机器人的手势指导并允许他们管理感兴趣的环境视觉信息,在探索灾后地区(防止倒塌,电气和有毒风险)时帮助救援人员并保护他们的完整性。因此,ARTU-R (A1救援任务UPM机器人)四足机器人已经配备了传感系统(激光雷达,热成像和RGB-D摄像机)来验证这一概念验证。另一方面,人机交互是通过使用Hololens眼镜来实现的。这项工作的主要贡献是基于ROS-Unity解决方案的混合现实系统的实施和评估,该系统能够在高水平上管理复杂腿机器人通过灾后环境的不同兴趣区域(由神经网络和视觉系统定义)的指导。机器人在每个访问点的主要任务包括通过热成像、RGB成像和神经网络检测受害者,并帮助受害者使用医疗设备。在再现灾后环境条件的情景(碎片、模拟受害者等)中进行了测试。与传统界面相比,使用沉浸式界面平均效率提高48%,时间优化21.4%。该方法已被证明可以提高救援人员控制复杂机器人系统的沉浸式体验。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Journal of Computational Design and Engineering
Journal of Computational Design and Engineering Computer Science-Human-Computer Interaction
CiteScore
7.70
自引率
20.40%
发文量
125
期刊介绍: Journal of Computational Design and Engineering is an international journal that aims to provide academia and industry with a venue for rapid publication of research papers reporting innovative computational methods and applications to achieve a major breakthrough, practical improvements, and bold new research directions within a wide range of design and engineering: • Theory and its progress in computational advancement for design and engineering • Development of computational framework to support large scale design and engineering • Interaction issues among human, designed artifacts, and systems • Knowledge-intensive technologies for intelligent and sustainable systems • Emerging technology and convergence of technology fields presented with convincing design examples • Educational issues for academia, practitioners, and future generation • Proposal on new research directions as well as survey and retrospectives on mature field.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信