{"title":"基于多信息融合和深度学习机器视觉的多关节机械臂故障检测与诊断","authors":"Jinghui Pan","doi":"10.1002/rob.22583","DOIUrl":null,"url":null,"abstract":"<div>\n \n <p>The multi-joint manipulator with vision sensors has been widely used in real applications. However, the fault detection and diagnosis accuracy are lowered and the time expense is increased for the increased number of sensors, as there are many factors that are relative with this problem. This paper is focused on the fault detection and diagnosis problem of multi-joint manipulator, and the problem was divided into two sub-problems. The first is that the position estimation strategy based on data fusion of visual sensor and the position sensor was designed to carry out the fault detection, and the whether the faults had happened or not were determined by the position estimation errors. The second was focused on the fault diagnosis problem, where the deep convolutional neural network (DCNN) fault diagnosis model based on time-frequency mixed signal was constructed. The proposed DCNN uses the time and frequency domain information as its inputs and executes the classification tasks. The specific fault was determined through the output of DCNN model. The DCNN model was activated only when the first fault detection unit indicated that there was a fault, so the time expense was reduced from 5.3 to 2.6 s. The experiment based on the AUBO-i5 manipulator was carried out to evaluate the proposed fault detection and diagnosis model, where 10 categories of data sets that represent different working conditions of manipulator were adopted. The experimental results showed that the proposed multi-joint manipulator fault detection could improve the position estimation accuracy by 41.2%, and the fault diagnosis accuracy was improved by 20%.</p>\n </div>","PeriodicalId":192,"journal":{"name":"Journal of Field Robotics","volume":"42 7","pages":"3308-3322"},"PeriodicalIF":5.2000,"publicationDate":"2025-05-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Fault Detection and Diagnosis of Multi-Joint Manipulator Based on Multi-Information Fusion and Deep-Learning Machine Vision\",\"authors\":\"Jinghui Pan\",\"doi\":\"10.1002/rob.22583\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div>\\n \\n <p>The multi-joint manipulator with vision sensors has been widely used in real applications. However, the fault detection and diagnosis accuracy are lowered and the time expense is increased for the increased number of sensors, as there are many factors that are relative with this problem. This paper is focused on the fault detection and diagnosis problem of multi-joint manipulator, and the problem was divided into two sub-problems. The first is that the position estimation strategy based on data fusion of visual sensor and the position sensor was designed to carry out the fault detection, and the whether the faults had happened or not were determined by the position estimation errors. The second was focused on the fault diagnosis problem, where the deep convolutional neural network (DCNN) fault diagnosis model based on time-frequency mixed signal was constructed. The proposed DCNN uses the time and frequency domain information as its inputs and executes the classification tasks. The specific fault was determined through the output of DCNN model. The DCNN model was activated only when the first fault detection unit indicated that there was a fault, so the time expense was reduced from 5.3 to 2.6 s. The experiment based on the AUBO-i5 manipulator was carried out to evaluate the proposed fault detection and diagnosis model, where 10 categories of data sets that represent different working conditions of manipulator were adopted. The experimental results showed that the proposed multi-joint manipulator fault detection could improve the position estimation accuracy by 41.2%, and the fault diagnosis accuracy was improved by 20%.</p>\\n </div>\",\"PeriodicalId\":192,\"journal\":{\"name\":\"Journal of Field Robotics\",\"volume\":\"42 7\",\"pages\":\"3308-3322\"},\"PeriodicalIF\":5.2000,\"publicationDate\":\"2025-05-05\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Field Robotics\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://onlinelibrary.wiley.com/doi/10.1002/rob.22583\",\"RegionNum\":2,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"ROBOTICS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Field Robotics","FirstCategoryId":"94","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/rob.22583","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ROBOTICS","Score":null,"Total":0}
Fault Detection and Diagnosis of Multi-Joint Manipulator Based on Multi-Information Fusion and Deep-Learning Machine Vision
The multi-joint manipulator with vision sensors has been widely used in real applications. However, the fault detection and diagnosis accuracy are lowered and the time expense is increased for the increased number of sensors, as there are many factors that are relative with this problem. This paper is focused on the fault detection and diagnosis problem of multi-joint manipulator, and the problem was divided into two sub-problems. The first is that the position estimation strategy based on data fusion of visual sensor and the position sensor was designed to carry out the fault detection, and the whether the faults had happened or not were determined by the position estimation errors. The second was focused on the fault diagnosis problem, where the deep convolutional neural network (DCNN) fault diagnosis model based on time-frequency mixed signal was constructed. The proposed DCNN uses the time and frequency domain information as its inputs and executes the classification tasks. The specific fault was determined through the output of DCNN model. The DCNN model was activated only when the first fault detection unit indicated that there was a fault, so the time expense was reduced from 5.3 to 2.6 s. The experiment based on the AUBO-i5 manipulator was carried out to evaluate the proposed fault detection and diagnosis model, where 10 categories of data sets that represent different working conditions of manipulator were adopted. The experimental results showed that the proposed multi-joint manipulator fault detection could improve the position estimation accuracy by 41.2%, and the fault diagnosis accuracy was improved by 20%.
期刊介绍:
The Journal of Field Robotics seeks to promote scholarly publications dealing with the fundamentals of robotics in unstructured and dynamic environments.
The Journal focuses on experimental robotics and encourages publication of work that has both theoretical and practical significance.