Robotics and Computer-integrated Manufacturing最新文献_第2页

Posture optimization for improving the kinematics performance of robotic polishing under combined constraints by using a KC-ADP method 基于KC-ADP方法的组合约束下机器人抛光运动学优化研究

IF 9.1 1区计算机科学

Robotics and Computer-integrated Manufacturing Pub Date : 2025-06-26 DOI: 10.1016/j.rcim.2025.103084

Yingpeng Wang, Huipeng Song, Haoyu Fu, Yuwen Sun

{"title":"Posture optimization for improving the kinematics performance of robotic polishing under combined constraints by using a KC-ADP method","authors":"Yingpeng Wang, Huipeng Song, Haoyu Fu, Yuwen Sun","doi":"10.1016/j.rcim.2025.103084","DOIUrl":"10.1016/j.rcim.2025.103084","url":null,"abstract":"<div><div>The flexibility of industrial robots in posture adjustment has driven their widespread adoption in polishing applications, providing an effective means to enhance machining performance. To satisfy the growing industrial demand for high-quality and high-efficiency machining of complex surfaces, posture optimization must address multiple constraints, including interference-free operation, singularity avoidance, stiffness performance index limits, and the kinematic parameter limits. The simultaneous consideration of all these factors poses a significant challenge, and existing methods do not adequately address the global optimality and the convergence of the solution process. This paper proposes a novel posture optimization model to improve the comprehensive performance of robotic postures by integrating enhancements in both kinematic and stiffness performance under fundamental geometric constraints. An efficient and stable algorithm, designated as Kinematics-Constrained Adaptive Dynamic Programming (KC-ADP), is developed to solve the optimization problem. First, the combined constraints are modeled based on the robotic polishing system. Next, feasible posture solutions corresponding to different redundant parameters are collected according to machining requirements and transformed into a directed graph using the proposed Multi-Constraints Search Space Generation (MCSSG) algorithm. The optimal posture sequence is then obtained through adaptive dynamic programming, ensuring the availability of feasible postures at each step and resolving the conflict between multi-order kinematic constraints and the objective function. A series of simulations and experiments were conducted to validate the proposed method and the results demonstrate that the proposed approach significantly improves machining performance.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"97 ","pages":"Article 103084"},"PeriodicalIF":9.1,"publicationDate":"2025-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144490009","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Robotic jamming-free and precise assembly for peg-in-hole operation by multi-DOF force-controlled parallel end-effector 采用多自由度力控并联末端执行器实现机器人无干扰、精密装配

IF 9.1 1区计算机科学

Robotics and Computer-integrated Manufacturing Pub Date : 2025-06-25 DOI: 10.1016/j.rcim.2025.103083

Qixiang Zhao , Hailemichael Yilma Hailegebrial , Wei Wang , Ke Wen , Lianyu Zheng

{"title":"Robotic jamming-free and precise assembly for peg-in-hole operation by multi-DOF force-controlled parallel end-effector","authors":"Qixiang Zhao , Hailemichael Yilma Hailegebrial , Wei Wang , Ke Wen , Lianyu Zheng","doi":"10.1016/j.rcim.2025.103083","DOIUrl":"10.1016/j.rcim.2025.103083","url":null,"abstract":"<div><div>In smart manufacturing, the precision peg-in-hole assembly under contact-unknown conditions poses a major challenge, which has spurred growing interest in enhancing active compliance in robotic assembly systems. Multi-Degree-of-Freedom (Multi-DOF) compliance provides high adaptability for addressing this assembly challenge. However, problems such as excessive size, insufficient dynamics, and complex jamming mechanisms have emerged. To solve these problems, we propose a jamming-free strategy. This strategy uses a lightweight, compact multi-DOF end-effector with a 3-Prismatic-Revolute-Spherical (3-PRS) parallel configuration, along with an active force control algorithm. The force control result of an absolute mean error of 1 N is observed for a setpoint value of 100 N along the z-axis. Meanwhile, the moments about the x- and y-axis are controlled within an absolute mean error of 0.04 N.mm. With the accurate compliance provided by the proposed force-controlled end-effector, the peg-in-hole operation is considered as a reciprocal transition between a transient one-point contact and a steady two-point jamming. Based on admittance control, a control strategy is designed to transform the two-point jamming into one-point contact, thereby preventing jamming during the precision peg-in-hole operation with significant uncertainties, eliminating the need for repeated positional adjustments during insertion. Experimental results show that the peg can be inserted into different holes with various pose errors and unknown dimensional tolerances. These results validate the rationality of the proposed jamming-free strategy for precision peg-in-hole assembly. The jamming-free peg-in-hole assembly employing a force-controlled parallel end-effector exhibits superior robustness and operational stability when it is applied into a satellite assembling task.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"97 ","pages":"Article 103083"},"PeriodicalIF":9.1,"publicationDate":"2025-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144471502","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Integrated dynamic scheduling method for hybrid flow shop with machine preventive maintenance based on cooperative multi-agent deep reinforcement learning 基于协同多智能体深度强化学习的混合流车间机器预防性维修集成动态调度方法

IF 9.1 1区计算机科学

Robotics and Computer-integrated Manufacturing Pub Date : 2025-06-25 DOI: 10.1016/j.rcim.2025.103085

Siqi Liu , Haiping Zhu , LieZheng Sheng

{"title":"Integrated dynamic scheduling method for hybrid flow shop with machine preventive maintenance based on cooperative multi-agent deep reinforcement learning","authors":"Siqi Liu , Haiping Zhu , LieZheng Sheng","doi":"10.1016/j.rcim.2025.103085","DOIUrl":"10.1016/j.rcim.2025.103085","url":null,"abstract":"<div><div>Hybrid flow shop widely used in manufacturing industry is facing the challenge of complex and dynamic production environment. Current study mostly cannot consider the machine preventive maintenance and dynamic events in hybrid flow shop scheduling process. Therefore, this paper presents an integrated dynamic scheduling method for hybrid flow shop scheduling problem- unrelated parallel machine considering preventive maintenance (DHSFP-UPM-PM). And the multi-scheduling objectives include minimizing completion time, processing cost and maintenance cost. Firstly, the definition of research problem and the basic maintenance strategy are presented in detail. And an integrated mathematic model of DHSFP-UPM-PM is constructed. Then the integrated dynamic scheduling framework based on cooperative multi-agent deep reinforcement learning is proposed to solve the DHSFP-UPM-PM. Based on the above, we proposed a cooperative multi-processing stage agents (PSA) approach to realize the transformation from traditional single-agent to multi-agent. Meanwhile, the cooperative multi-agent Markova Decision Process is formulated to clarify the interaction between each agent and production environment. The state and action space as the key elements of scheduling model is also designed for each PSA. To optimize scheduling objectives, this paper further formulates new global reward mechanism and centralized training-decentralized execution method based on multi agent proximal policy optimization. Lastly, the experiment results verify the superiority and effectiveness of the proposed method when solving integrated scheduling problem and dynamic event. And the proposed method presents remarkable adaptability and flexibility under a different production scenario which prove the benefits of multi-agent deep reinforcement learning in complex and dynamic environment.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"97 ","pages":"Article 103085"},"PeriodicalIF":9.1,"publicationDate":"2025-06-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144472090","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Autogeneration and optimization of pick-and-place trajectories in robotic systems: A data-driven approach 机器人系统中拾取轨迹的自动生成和优化：数据驱动的方法

IF 9.1 1区计算机科学

Robotics and Computer-integrated Manufacturing Pub Date : 2025-06-24 DOI: 10.1016/j.rcim.2025.103080

Mingkun Wu , Alisa Rupenyan , Burkhard Corves

{"title":"Autogeneration and optimization of pick-and-place trajectories in robotic systems: A data-driven approach","authors":"Mingkun Wu , Alisa Rupenyan , Burkhard Corves","doi":"10.1016/j.rcim.2025.103080","DOIUrl":"10.1016/j.rcim.2025.103080","url":null,"abstract":"<div><div>For manufacturing processes in industries such as aerospace, automotive and electronics, it is essential for robots to perform pick-and-place tasks with high efficiency and accuracy. To this end, we propose a data-driven framework to generate a pick-and-place trajectory that ensures high-accuracy tracking while simultaneously reducing residual vibration, which is particularly valuable for commercial industrial robots with unchangeable control systems. The proposed approach includes both the trajectory generation and the trajectory compensation phases. In the first phase, we plan a pick-and-place trajectory that effectively attenuate residual vibration by minimizing the acceleration energy within a specific frequency spectrum, where the frequency parameters and the time ratio are tuned by Bayesian optimization. In the second phase, we focus on improving the tracking accuracy by incorporating a trajectory compensation term. More precisely, we first learn a Koopman operator-based linear predictor, where a model-agnostic meta-learning framework is introduced to mitigate the demand for massive data from the target system. Then, we calculate the trajectory compensation term using an iterative learning control-based method. The proposed methodology is entirely data driven, enabling its application in various robotic systems and has potential in other manufacturing applications. We demonstrate the approach through high-fidelity simulations on Delta robots – a representative parallel robot, where trajectory generation effectively removes vibrations, and through physical experiments on UR5 robots – a typical serial robot. The results of the experiment show that the positioning accuracy of the three joints of the UR5 robot improved by 94%, 43%, and 96%.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"97 ","pages":"Article 103080"},"PeriodicalIF":9.1,"publicationDate":"2025-06-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144365296","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

An adaptive disturbance compensation method for force-sensorless control systems applied to robotic milling 应用于铣削机器人无力传感器控制系统的自适应扰动补偿方法

IF 9.1 1区计算机科学

Robotics and Computer-integrated Manufacturing Pub Date : 2025-06-19 DOI: 10.1016/j.rcim.2025.103082

Han-Hao Tsai, Jen-Yuan Chang

{"title":"An adaptive disturbance compensation method for force-sensorless control systems applied to robotic milling","authors":"Han-Hao Tsai, Jen-Yuan Chang","doi":"10.1016/j.rcim.2025.103082","DOIUrl":"10.1016/j.rcim.2025.103082","url":null,"abstract":"<div><div>This paper introduces a novel disturbance compensation method for a robust, active-force-controlled end-effector. This system integrates a disturbance observer (DOB) and a reaction force observer (RFOB) while employing a numerical optimization-based extremum-seeking algorithm. Conventional force/torque sensors, when employed in serially connected manipulator systems, often trigger unstable responses due to the presence of non-collocated modes. Furthermore, conventional reaction force observers may fail to accurately estimate the contact force between the robot and its environment when external disturbance terms are not perfectly modeled a priori. In response to these challenges, recent research has reintegrated force/torque sensors into the control architecture, employing filter-based sensor fusion techniques to mitigate disturbance effects. However, these approaches fail to address the inherent stability challenges caused by the mounting and serial connection of force/torque sensors within the robot manipulator system, which in turn increases the design complexity of reaction-force-observer-based force control systems. To overcome these limitations, this paper proposes adaptive disturbance compensation methods that leverage position feedback information related to the depth of cut in milling processes. The proposed method adaptively compensates for time-varying disturbances, such as tool wear and abrupt feed rate changes, ensuring consistent performance under dynamic conditions. Compared to conventional position-controlled and force-controlled methods, the proposed approach exhibits improved robustness, precision, and versatility, positioning it as a promising solution for advancing robotic milling technologies toward practical industrial applications.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"97 ","pages":"Article 103082"},"PeriodicalIF":9.1,"publicationDate":"2025-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144313478","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Digital-twin-assisted multi-stage machining of thin-wall structures using interchangeable robotic and human-assisted automation 使用可互换的机器人和人工辅助自动化的薄壁结构的数字双辅助多级加工

IF 9.1 1区计算机科学

Robotics and Computer-integrated Manufacturing Pub Date : 2025-06-17 DOI: 10.1016/j.rcim.2025.103077

Mariam Abed , Abdelkhalick Mohammad , Dragos Axinte , Andres Gameros , David Askew

{"title":"Digital-twin-assisted multi-stage machining of thin-wall structures using interchangeable robotic and human-assisted automation","authors":"Mariam Abed , Abdelkhalick Mohammad , Dragos Axinte , Andres Gameros , David Askew","doi":"10.1016/j.rcim.2025.103077","DOIUrl":"10.1016/j.rcim.2025.103077","url":null,"abstract":"<div><div>Interconnected intelligent systems in multi-stage smart machining environments are an advancing area of research, demonstrating many real-life opportunities that can benefit from the development and integration of cyber-physical systems into machining habitats, while different automation levels in industrial manufacturing sites call for flexibility of core strategies towards smart machining ecosystems. This article introduces a versatile and smart multi-stage machining environment for the controlled clamping and machining of low-rigidity structures in an interconnected cyber-physical factory. This is exemplified by a deformation-prone thin-wall workpiece, which undergoes controlled clamping, enabled by interchangeable robotic automation and automation via human-cyber-physical systems, as well as digital-twin-assisted corrective machining enabled by the swift estimation of workpiece deformations and multi-stage communication between machining habitats. The underlying digital twin presents a fast, lightweight simulation approach, based on a mass-spring-lattice model, allowing information flow from and to systems, which is utilized by the CNC machine as well as the interchangeable robot- and human-in-the-loop clamping enablers. By employing this controlled clamping approach workpiece deformations are aimed to be minimized. At the same time, a desired total clamping force is achieved in order to perform subsequent digital-twin-assisted machining corrections to reduce deformation-caused flatness errors. Ultimately, this article presents an intelligent multi-stage machining scenario where digital-twin enabled information moves along with thin-wall structures and branches out for knowledge-based control and corrections to robots, humans and CNC machines respectively, showcasing a real-life example for versatile, information-driven smart machining ecosystems.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"97 ","pages":"Article 103077"},"PeriodicalIF":9.1,"publicationDate":"2025-06-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144296947","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A green and efficient disassembly line balancing with human-robot collaboration and destructive disassembly 一条人机协作与破坏性拆卸相平衡的绿色高效装配线

IF 9.1 1区计算机科学

Robotics and Computer-integrated Manufacturing Pub Date : 2025-06-16 DOI: 10.1016/j.rcim.2025.103081

Wei Zhang , Yibing Li , Kaipu Wang , Wenjun Xu , Liang Gao

{"title":"A green and efficient disassembly line balancing with human-robot collaboration and destructive disassembly","authors":"Wei Zhang , Yibing Li , Kaipu Wang , Wenjun Xu , Liang Gao","doi":"10.1016/j.rcim.2025.103081","DOIUrl":"10.1016/j.rcim.2025.103081","url":null,"abstract":"<div><div>Human-robot collaboration combines the strengths of both humans and robots to enhance disassembly line efficiency. Considering the indivisibility and recovery value of certain components, this study incorporates destructive disassembly into a human-robot collaborative disassembly line. A mixed-integer linear programming model of the disassembly line balancing problem is constructed. The model accounts for task precedence relationships, task attributes, disassembly modes, human-robot collaboration, and the configuration of humans and robots. The objective is to minimize cycle time, smoothness index, and disassembly energy consumption while maximizing disassembly profit. The algorithm uses a three-layer encoding strategy based on task sequence, task operators, and disassembly modes, with an optimization-driven initialization to improve the initial solution quality. Five selection strategies and two neighborhood search strategies are designed, and during the iterative process, the strategy is dynamically adjusted through Q-learning to enhance both global search and local search capabilities. The effectiveness and superiority of the proposed algorithm are validated through three types of test case experiments, compared with the five latest algorithms. Finally, the model and algorithm are applied to a real-world laptop disassembly case. The results show that the introduction of collaborative robots in disassembly significantly reduces disassembly costs. Compared to manual disassembly, the cycle time of the disassembly line can be reduced by 24.39%, and idle time can be reduced by 33.64% in the human-robot collaborative disassembly mode. Compared to non-destructive disassembly, destructive disassembly can reduce energy consumption by 28.20%.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"97 ","pages":"Article 103081"},"PeriodicalIF":9.1,"publicationDate":"2025-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144291028","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Coating defect detection in intelligent manufacturing: Advances, challenges, and future trends 智能制造中的涂层缺陷检测：进展、挑战和未来趋势

IF 9.1 1区计算机科学

Robotics and Computer-integrated Manufacturing Pub Date : 2025-06-14 DOI: 10.1016/j.rcim.2025.103079

Bin Zi , Kai Tang , Yuan Li , Kai Feng , Yongkui Liu , Lihui Wang

{"title":"Coating defect detection in intelligent manufacturing: Advances, challenges, and future trends","authors":"Bin Zi , Kai Tang , Yuan Li , Kai Feng , Yongkui Liu , Lihui Wang","doi":"10.1016/j.rcim.2025.103079","DOIUrl":"10.1016/j.rcim.2025.103079","url":null,"abstract":"<div><div>Spraying is a critical surface treatment process in intelligent manufacturing, and coating quality directly affects product performance. Therefore, efficient, accurate, and intelligent coating defect detection is an essential technique to ensure product reliability. The past decade has witnessed rapid progress in coating defect detection techniques. However, most existing studies have focused on specific methods or application scenarios, and there is a lack of systematic reviews that provide a comprehensive overview of this particular research area. To fill this research gap, this paper systematically reviews recent advances in coating defect detection, which covers methods from physical property-based non-destructive testing to deep learning-based approaches. Their fundamental principles, applicability in intelligent manufacturing, and current research progress are examined, and key challenges and potential solutions are discussed. Furthermore, integration of advanced intelligent manufacturing technologies into coating defect detection systems is analyzed to enhance system-level digitalization, automation, and efficiency. Finally, future development trends are explored and analyzed, including collaborative perception, cross-modal fusion, and autonomous decision-making. It is expected that this review will help to advance and accelerate theoretical research and engineering applications in coating defect detection by providing researchers with a comprehensive understanding.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"97 ","pages":"Article 103079"},"PeriodicalIF":9.1,"publicationDate":"2025-06-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144288823","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Large and small-scale models’ fusion-driven proactive robotic manipulation control for human-robot collaborative assembly in industry 5.0 面向工业5.0人机协同装配的大、小模型融合驱动主动机器人操作控制

IF 9.1 1区计算机科学

Robotics and Computer-integrated Manufacturing Pub Date : 2025-06-12 DOI: 10.1016/j.rcim.2025.103078

Dongxu Ma , Chao Zhang , Qingfeng Xu , Guanghui Zhou

{"title":"Large and small-scale models’ fusion-driven proactive robotic manipulation control for human-robot collaborative assembly in industry 5.0","authors":"Dongxu Ma , Chao Zhang , Qingfeng Xu , Guanghui Zhou","doi":"10.1016/j.rcim.2025.103078","DOIUrl":"10.1016/j.rcim.2025.103078","url":null,"abstract":"<div><div>Human-robot collaborative (HRC) assembly has been popular by combining human creativity and dexterity with robotic precision for higher assembly efficiency and resilience in industry 5.0. Nevertheless, current HRC assembly systems rely predefined codes, limiting robot adaptability to dynamic and unstructured assembly environments. To bridge the gap, this paper proposes a novel proactive robotic manipulation control method for HRC assembly, which fully utilizes large-scale model (LSM) in cognitive computing and reasoning for dynamic robotic control path planning, and small-scale models (SSMs) in efficiently computing for dynamic robotic control demand perception and control constraints verification. Specifically, LSM, namely ChatGPT 4o, is deployed on the cloud to proactively generate robotic control constraints according to the robotic control demand derived from SSMs on the edge. Here, two kinds of SSMs are developed, including robotic control demands perception model and robotic control constraints verification model. For robotic control demands perception, an ensemble encoder model is proposed for ongoing human assembly action detection, on which a vision model and fine-tuned assembly instruction generation model are designed for assembly manipulation keypoints image and robot control instruction generation, serving as the input for LSM. For robotic control constraints verification, a digital twin model is used to verify the control constraints derived from LSM, where verified constraints are used for robotic control during assembly process. Finally, the feasibility and effectiveness of the proposed approach are demonstrated through experiments on an HRC assembly process, where over 99 % accuracy for human assembly action detection and 80 % task execution accuracy are conducted.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"97 ","pages":"Article 103078"},"PeriodicalIF":9.1,"publicationDate":"2025-06-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144271506","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Multi-modal digital twins for industrial anomaly detection: Framework, method, and application 工业异常检测的多模态数字孪生：框架、方法和应用

IF 9.1 1区计算机科学

Robotics and Computer-integrated Manufacturing Pub Date : 2025-06-12 DOI: 10.1016/j.rcim.2025.103068

Cheng Ren , Ming Li , Cailian Chen , Xinping Guan , George Q. Huang

{"title":"Multi-modal digital twins for industrial anomaly detection: Framework, method, and application","authors":"Cheng Ren , Ming Li , Cailian Chen , Xinping Guan , George Q. Huang","doi":"10.1016/j.rcim.2025.103068","DOIUrl":"10.1016/j.rcim.2025.103068","url":null,"abstract":"<div><div>Anomaly detection plays a key role in maintaining the reliable and stable operation of industrial systems, especially in high-reliability fields. Conventional single-modal data cannot provide comprehensive information about the detected object, resulting in false or missed detection. To address the challenges of complex anomaly patterns and heterogeneous data in industrial scenarios, we propose MMDT-IAD, a multi-modal digital twin (DT)-based anomaly detection framework that integrates edge–cloud collaboration. By lever- aging physical, geometric, visual, and semantic modalities, MMDT-IAD constructs a comprehensive virtual representation of monitored objects and enables real-time, scalable detection across distributed industrial environments. Next, to enable efficient fusion of heterogeneous DT modalities, we propose a One-Primary- Three-Auxiliary (1P3A) cross-modal decision fusion strategy. Finally, we apply the MMDT-IAD frame-work to the anomaly detection of aviation electrical connector pins, and present a detailed application process. The experimental results prove the effectiveness of the MMDT-IAD framework in detecting abnormal pins. Moreover, we discuss the generality of MMDT-IAD framework considering several common industrial anomalies. These results highlight the potential of MMDT-IAD framework and 1P3A method to significantly improve anomaly detection in other complex industrial scenarios.</div></div>","PeriodicalId":21452,"journal":{"name":"Robotics and Computer-integrated Manufacturing","volume":"97 ","pages":"Article 103068"},"PeriodicalIF":9.1,"publicationDate":"2025-06-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144264044","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0