Synchronized Control of Robotic Arm based on Virtual Reality and IMU Sensing

Q3 Mathematics

WSEAS Transactions on Systems and Control Pub Date : 2023-12-14 DOI:10.37394/23203.2023.18.49

Chih-Jer Lin, Ting-Yi Sie, Yu-Sheng Chang

{"title":"Synchronized Control of Robotic Arm based on Virtual Reality and IMU Sensing","authors":"Chih-Jer Lin, Ting-Yi Sie, Yu-Sheng Chang","doi":"10.37394/23203.2023.18.49","DOIUrl":null,"url":null,"abstract":"This study introduces a robotic control system that combines virtual reality integration and inertial measurement units (IMU) using mixed reality (MR) devices. The system integrates three main modules: (1) virtual reality (VR), which simulates remote reality by the Hololens2, (2) the wearable IMUs device, which captures the operator's hand movements; and (3) the robotic arm UR5, which is controlled by the user and the VR environment. The virtual and physical systems communicate via a Message Queuing Telemetry Transport (MQQT) communication architecture to establish communication between modules. To introduce a closed-loop control system for the robotic arm, model predictive control (MPC) was achieved with precise path planning to provide a flexible, intuitive, and reliable method to operate the remote-controlled manipulator. To validate the system integration and functions, two demonstrations were conducted: (a) the offline mode, where the VR module of the robotic arm was controlled by the IMUs device to check correctness, and (b) the online mode, where the control command was transferred to UR5 to complete a target mission via artificial potential field (APF) adjustment. The primary outcome of this study was the development of virtual and real industrial robotic arms to test the developed model in VR and shop floor labs.","PeriodicalId":39422,"journal":{"name":"WSEAS Transactions on Systems and Control","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2023-12-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"WSEAS Transactions on Systems and Control","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.37394/23203.2023.18.49","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"Mathematics","Score":null,"Total":0}

引用次数: 0

Abstract

This study introduces a robotic control system that combines virtual reality integration and inertial measurement units (IMU) using mixed reality (MR) devices. The system integrates three main modules: (1) virtual reality (VR), which simulates remote reality by the Hololens2, (2) the wearable IMUs device, which captures the operator's hand movements; and (3) the robotic arm UR5, which is controlled by the user and the VR environment. The virtual and physical systems communicate via a Message Queuing Telemetry Transport (MQQT) communication architecture to establish communication between modules. To introduce a closed-loop control system for the robotic arm, model predictive control (MPC) was achieved with precise path planning to provide a flexible, intuitive, and reliable method to operate the remote-controlled manipulator. To validate the system integration and functions, two demonstrations were conducted: (a) the offline mode, where the VR module of the robotic arm was controlled by the IMUs device to check correctness, and (b) the online mode, where the control command was transferred to UR5 to complete a target mission via artificial potential field (APF) adjustment. The primary outcome of this study was the development of virtual and real industrial robotic arms to test the developed model in VR and shop floor labs.

查看原文本刊更多论文

基于虚拟现实和 IMU 传感的机械臂同步控制

本研究介绍了一种利用混合现实（MR）设备将虚拟现实集成与惯性测量单元（IMU）相结合的机器人控制系统。该系统集成了三个主要模块：(1) 虚拟现实（VR），通过 Hololens2 模拟远程现实；(2) 可穿戴式 IMUs 设备，捕捉操作员的手部动作；(3) 机械臂 UR5，由用户和 VR 环境控制。虚拟系统和物理系统通过消息队列遥测传输（MQQT）通信架构建立模块之间的通信。为了给机械臂引入闭环控制系统，通过精确的路径规划实现了模型预测控制（MPC），为遥控机械手的操作提供了灵活、直观和可靠的方法。为了验证系统的集成和功能，进行了两次演示：（a）离线模式，通过 IMUs 设备控制机械臂的 VR 模块，以检查正确性；（b）在线模式，通过人工势场（APF）调整，将控制命令传输到 UR5，以完成目标任务。这项研究的主要成果是开发了虚拟和真实的工业机械臂，以便在虚拟现实和车间实验室测试所开发的模型。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

WSEAS Transactions on Systems and Control Mathematics-Control and Optimization

CiteScore

1.80

自引率

0.00%

发文量

期刊介绍： WSEAS Transactions on Systems and Control publishes original research papers relating to systems theory and automatic control. We aim to bring important work to a wide international audience and therefore only publish papers of exceptional scientific value that advance our understanding of these particular areas. The research presented must transcend the limits of case studies, while both experimental and theoretical studies are accepted. It is a multi-disciplinary journal and therefore its content mirrors the diverse interests and approaches of scholars involved with systems theory, dynamical systems, linear and non-linear control, intelligent control, robotics and related areas. We also welcome scholarly contributions from officials with government agencies, international agencies, and non-governmental organizations.