3D guiding assisted augmented assembly technology with rapid object detection in dynamic environment

IF 8 1区工程技术 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Advanced Engineering Informatics Pub Date : 2024-10-01 DOI:10.1016/j.aei.2024.102857

Chengshun Li, Xiaonan Yang, Yaoguang Hu, Shangsi Wu, Jingfei Wang, Peng Wang

{"title":"3D guiding assisted augmented assembly technology with rapid object detection in dynamic environment","authors":"Chengshun Li, Xiaonan Yang, Yaoguang Hu, Shangsi Wu, Jingfei Wang, Peng Wang","doi":"10.1016/j.aei.2024.102857","DOIUrl":null,"url":null,"abstract":"<div><div>In the field of industrial assembly, augmented reality (AR) technology has played an important role and demonstrated its enormous development potential in the future. With the current development of product assembly towards customization and diversification, it is difficult to meet the requirements of augmented assembly (AA) by relying on static instructions registered with markers. However, most augmented assembly guidance systems used for dynamic environments are complex, cumbersome, and exhibit high latency, significantly impacting the user experience. In addition, the narrow field of view (Fov) of AR glasses also limits its further application in industrial scene. In response to the above issues, this article proposes an improved 3D guiding assisted augmented assembly technology. Firstly, a lightweight model Yolov7-Slim is proposed to achieve object detection on 2D images, which reduces File size by 26.7 % and improves running speed by 15.3 % compared to the Yolov7-tiny model. Secondly, a 3D positioning algorithm is proposed to achieve the rapid conversion of 2D coordinates to 3D coordinates. Finally, a user-oriented two-stage guidance mechanism is designed to compensate for the limitation of the narrow Fov of AR glasses. To quantify the performance of proposed technology, a 3D guiding assisted augmented assembly system (3DG3AS) was developed and validated in a reducer assembly experiment.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"62 ","pages":"Article 102857"},"PeriodicalIF":8.0000,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Advanced Engineering Informatics","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1474034624005056","RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}

引用次数: 0

Abstract

In the field of industrial assembly, augmented reality (AR) technology has played an important role and demonstrated its enormous development potential in the future. With the current development of product assembly towards customization and diversification, it is difficult to meet the requirements of augmented assembly (AA) by relying on static instructions registered with markers. However, most augmented assembly guidance systems used for dynamic environments are complex, cumbersome, and exhibit high latency, significantly impacting the user experience. In addition, the narrow field of view (Fov) of AR glasses also limits its further application in industrial scene. In response to the above issues, this article proposes an improved 3D guiding assisted augmented assembly technology. Firstly, a lightweight model Yolov7-Slim is proposed to achieve object detection on 2D images, which reduces File size by 26.7 % and improves running speed by 15.3 % compared to the Yolov7-tiny model. Secondly, a 3D positioning algorithm is proposed to achieve the rapid conversion of 2D coordinates to 3D coordinates. Finally, a user-oriented two-stage guidance mechanism is designed to compensate for the limitation of the narrow Fov of AR glasses. To quantify the performance of proposed technology, a 3D guiding assisted augmented assembly system (3DG3AS) was developed and validated in a reducer assembly experiment.

查看原文本刊更多论文

在动态环境中快速检测物体的 3D 导向辅助增强装配技术

在工业装配领域，增强现实（AR）技术发挥了重要作用，并展示了其未来巨大的发展潜力。当前，产品装配正朝着定制化和多样化方向发展，仅靠标记注册的静态指令已难以满足增强装配（AA）的要求。然而，大多数用于动态环境的增强装配引导系统都非常复杂、繁琐，而且延迟较高，严重影响了用户体验。此外，AR 眼镜的视野（Fov）狭窄也限制了其在工业场景中的进一步应用。针对上述问题，本文提出了一种改进的 3D 导向辅助增强装配技术。首先，提出了一种轻量级模型 Yolov7-Slim 来实现二维图像上的物体检测，与 Yolov7-tiny 模型相比，文件大小减少了 26.7%，运行速度提高了 15.3%。其次，提出了一种三维定位算法，以实现二维坐标到三维坐标的快速转换。最后，设计了一种面向用户的两阶段引导机制，以弥补 AR 眼镜视场角窄的限制。为了量化所提技术的性能，我们开发了三维导向辅助增强装配系统（3DG3AS），并在减速器装配实验中进行了验证。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Advanced Engineering Informatics 工程技术-工程：综合

CiteScore

12.40

自引率

18.20%

发文量

292

审稿时长

45 days

期刊介绍： Advanced Engineering Informatics is an international Journal that solicits research papers with an emphasis on 'knowledge' and 'engineering applications'. The Journal seeks original papers that report progress in applying methods of engineering informatics. These papers should have engineering relevance and help provide a scientific base for more reliable, spontaneous, and creative engineering decision-making. Additionally, papers should demonstrate the science of supporting knowledge-intensive engineering tasks and validate the generality, power, and scalability of new methods through rigorous evaluation, preferably both qualitatively and quantitatively. Abstracting and indexing for Advanced Engineering Informatics include Science Citation Index Expanded, Scopus and INSPEC.