由自主机器人驱动的基于视觉人工智能的人机协作装配

IF 3.2 3区工程技术 Q2 ENGINEERING, INDUSTRIAL

Cirp Annals-Manufacturing Technology Pub Date : 2024-01-01 DOI:10.1016/j.cirp.2024.03.004

{"title":"由自主机器人驱动的基于视觉人工智能的人机协作装配","authors":"","doi":"10.1016/j.cirp.2024.03.004","DOIUrl":null,"url":null,"abstract":"<div><p>Autonomous robots that understand human instructions can significantly enhance the efficiency in human-robot assembly operations where robotic support is needed to handle unknown objects and/or provide on-demand assistance. This paper introduces a vision AI-based method for human-robot collaborative (HRC) assembly, enabled by a large language model (LLM). Upon 3D object reconstruction and pose establishment through neural object field modelling, a visual servoing-based mobile robotic system performs object manipulation and navigation guidance to a mobile robot. The LLM model provides text-based logic reasoning and high-level control command generation for natural human-robot interactions. The effectiveness of the presented method is experimentally demonstrated.</p></div>","PeriodicalId":55256,"journal":{"name":"Cirp Annals-Manufacturing Technology","volume":"73 1","pages":"Pages 13-16"},"PeriodicalIF":3.2000,"publicationDate":"2024-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0007850624000052/pdfft?md5=62e8d0f9fca044ef8672b7dddb628301&pid=1-s2.0-S0007850624000052-main.pdf","citationCount":"0","resultStr":"{\"title\":\"Vision AI-based human-robot collaborative assembly driven by autonomous robots\",\"authors\":\"\",\"doi\":\"10.1016/j.cirp.2024.03.004\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><p>Autonomous robots that understand human instructions can significantly enhance the efficiency in human-robot assembly operations where robotic support is needed to handle unknown objects and/or provide on-demand assistance. This paper introduces a vision AI-based method for human-robot collaborative (HRC) assembly, enabled by a large language model (LLM). Upon 3D object reconstruction and pose establishment through neural object field modelling, a visual servoing-based mobile robotic system performs object manipulation and navigation guidance to a mobile robot. The LLM model provides text-based logic reasoning and high-level control command generation for natural human-robot interactions. The effectiveness of the presented method is experimentally demonstrated.</p></div>\",\"PeriodicalId\":55256,\"journal\":{\"name\":\"Cirp Annals-Manufacturing Technology\",\"volume\":\"73 1\",\"pages\":\"Pages 13-16\"},\"PeriodicalIF\":3.2000,\"publicationDate\":\"2024-01-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.sciencedirect.com/science/article/pii/S0007850624000052/pdfft?md5=62e8d0f9fca044ef8672b7dddb628301&pid=1-s2.0-S0007850624000052-main.pdf\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Cirp Annals-Manufacturing Technology\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0007850624000052\",\"RegionNum\":3,\"RegionCategory\":\"工程技术\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"ENGINEERING, INDUSTRIAL\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Cirp Annals-Manufacturing Technology","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0007850624000052","RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ENGINEERING, INDUSTRIAL","Score":null,"Total":0}

引用次数: 0

摘要

能够理解人类指令的自主机器人可以显著提高人机装配作业的效率，在这种情况下，需要机器人支持来处理未知物体和/或提供按需协助。本文介绍了一种基于视觉人工智能的人机协作（HRC）装配方法，该方法由大型语言模型（LLM）支持。通过神经对象场建模进行三维对象重建和姿态建立后，基于视觉伺服的移动机器人系统将执行对象操作并为移动机器人提供导航指导。大语言模型提供了基于文本的逻辑推理和高级控制指令生成，以实现自然的人机交互。实验证明了该方法的有效性。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Vision AI-based human-robot collaborative assembly driven by autonomous robots

Autonomous robots that understand human instructions can significantly enhance the efficiency in human-robot assembly operations where robotic support is needed to handle unknown objects and/or provide on-demand assistance. This paper introduces a vision AI-based method for human-robot collaborative (HRC) assembly, enabled by a large language model (LLM). Upon 3D object reconstruction and pose establishment through neural object field modelling, a visual servoing-based mobile robotic system performs object manipulation and navigation guidance to a mobile robot. The LLM model provides text-based logic reasoning and high-level control command generation for natural human-robot interactions. The effectiveness of the presented method is experimentally demonstrated.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Cirp Annals-Manufacturing Technology 工程技术-工程：工业

CiteScore

7.50

自引率

9.80%

发文量

137

审稿时长

13.5 months

期刊介绍： CIRP, The International Academy for Production Engineering, was founded in 1951 to promote, by scientific research, the development of all aspects of manufacturing technology covering the optimization, control and management of processes, machines and systems. This biannual ISI cited journal contains approximately 140 refereed technical and keynote papers. Subject areas covered include: Assembly, Cutting, Design, Electro-Physical and Chemical Processes, Forming, Abrasive processes, Surfaces, Machines, Production Systems and Organizations, Precision Engineering and Metrology, Life-Cycle Engineering, Microsystems Technology (MST), Nanotechnology.