Rescheduling human-robot collaboration tasks under dynamic disassembly scenarios: An MLLM-KG collaboratively enabled approach

Abstract

During product recycling, the uncertainty of the degradation level of end-of-life products leads to dynamic conditions such as component corrosion and damage during the disassembly process. Therefore, enhancing the robot's perception of disassembly scenarios and matching historical disassembly experiences is crucial for task rescheduling in human-robot collaborative disassembly (HRCD) under dynamic conditions. To address this, this paper proposes a dynamic task rescheduling method for human-robot collaborative disassembly, empowered by the synergy of Knowledge Graph (KG) and Multimodal Large Language Model (MLLM). Leveraging a Mark-Aware image preprocessing module and prompt-based scene understanding, the physical characteristics and occlusion relationships of disassembly targets are extracted. The concept of affordance is introduced, and an Affordance KG is constructed to recommend disassembly actions based on the physical features of objects in the scene. A task allocation standard for human-robot collaboration is designed, which, combined with depth and human factor information from mixed reality scenarios, enables dynamic task rescheduling and reconstruction of the entire human-robot collaborative disassembly process. The proposed method is validated through a case study on human-robot collaborative disassembly of end-of-life automotive lithium-ion batteries. Experimental results demonstrate that the method exhibits strong robustness and generalizability in dynamic disassembly scenarios, accurately identifying the physical features of components and recommending appropriate disassembly actions under conditions such as component corrosion, damage, and tool unavailability, thus achieving effective task rescheduling.