Visual control of a cable-driven flexible robotic arm with a spinal structure based on video understanding

IF 5.4 2区计算机科学 Q1 AUTOMATION & CONTROL SYSTEMS

Control Engineering Practice Pub Date : 2025-03-24 DOI:10.1016/j.conengprac.2025.106303

Xiang Zhang , Xuesong Wu , Kangjia Fu , Chong Sun , Sunquan Yu , Qi Zhang , Teng Yi , Xiaoqian Chen

{"title":"Visual control of a cable-driven flexible robotic arm with a spinal structure based on video understanding","authors":"Xiang Zhang , Xuesong Wu , Kangjia Fu , Chong Sun , Sunquan Yu , Qi Zhang , Teng Yi , Xiaoqian Chen","doi":"10.1016/j.conengprac.2025.106303","DOIUrl":null,"url":null,"abstract":"<div><div>Flexible robotic arms have higher degrees of freedom, making them increasingly popular in tasks such as grasping in narrow environments. However, their high-compliance characteristics and occlusion in the environment pose considerable challenges in the precise control consistent with human requirements. This study combines vision-based knowledge representation with a large language model to help a cable-driven flexible robotic arm with a spinal structure better understand human intentions and mimic human actions. In particular, a visual servo system closely coupled with the flexible robotic arm is designed, which can effectively reduce the impact of occlusion on visual positioning. In a narrow experimental environment, the recognition accuracy of the coupled visual dynamic adjustment system improved by 34.8% compared with relying solely on visual recognition from the end of the arm, and by 28.7% compared with using the external camera visual recognition alone. Subsequently, aimed to perform fine manipulations of the flexible robotic arm, a data-driven nonlinear modeling method is proposed and a coarse-to-fine visual grasping control system is designed. Experiments across eight task scenarios validate the precise control and interactivity of the system in narrow environments using a flexible robotic arm with a spinal structure.</div></div>","PeriodicalId":50615,"journal":{"name":"Control Engineering Practice","volume":"160 ","pages":"Article 106303"},"PeriodicalIF":5.4000,"publicationDate":"2025-03-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Control Engineering Practice","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0967066125000668","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"AUTOMATION & CONTROL SYSTEMS","Score":null,"Total":0}

引用次数: 0

Abstract

Flexible robotic arms have higher degrees of freedom, making them increasingly popular in tasks such as grasping in narrow environments. However, their high-compliance characteristics and occlusion in the environment pose considerable challenges in the precise control consistent with human requirements. This study combines vision-based knowledge representation with a large language model to help a cable-driven flexible robotic arm with a spinal structure better understand human intentions and mimic human actions. In particular, a visual servo system closely coupled with the flexible robotic arm is designed, which can effectively reduce the impact of occlusion on visual positioning. In a narrow experimental environment, the recognition accuracy of the coupled visual dynamic adjustment system improved by 34.8% compared with relying solely on visual recognition from the end of the arm, and by 28.7% compared with using the external camera visual recognition alone. Subsequently, aimed to perform fine manipulations of the flexible robotic arm, a data-driven nonlinear modeling method is proposed and a coarse-to-fine visual grasping control system is designed. Experiments across eight task scenarios validate the precise control and interactivity of the system in narrow environments using a flexible robotic arm with a spinal structure.

查看原文本刊更多论文

求助全文

约1分钟内获得全文求助全文

来源期刊

Control Engineering Practice 工程技术-工程：电子与电气

CiteScore

9.20

自引率

12.20%

发文量

183

审稿时长

44 days

期刊介绍： Control Engineering Practice strives to meet the needs of industrial practitioners and industrially related academics and researchers. It publishes papers which illustrate the direct application of control theory and its supporting tools in all possible areas of automation. As a result, the journal only contains papers which can be considered to have made significant contributions to the application of advanced control techniques. It is normally expected that practical results should be included, but where simulation only studies are available, it is necessary to demonstrate that the simulation model is representative of a genuine application. Strictly theoretical papers will find a more appropriate home in Control Engineering Practice''s sister publication, Automatica. It is also expected that papers are innovative with respect to the state of the art and are sufficiently detailed for a reader to be able to duplicate the main results of the paper (supplementary material, including datasets, tables, code and any relevant interactive material can be made available and downloaded from the website). The benefits of the presented methods must be made very clear and the new techniques must be compared and contrasted with results obtained using existing methods. Moreover, a thorough analysis of failures that may happen in the design process and implementation can also be part of the paper. The scope of Control Engineering Practice matches the activities of IFAC. Papers demonstrating the contribution of automation and control in improving the performance, quality, productivity, sustainability, resource and energy efficiency, and the manageability of systems and processes for the benefit of mankind and are relevant to industrial practitioners are most welcome.