Dynamic scenario-enhanced diverse human motion prediction network for proactive human–robot collaboration in customized assembly tasks

Abstract

Human motion prediction is crucial for facilitating human–robot collaboration in customized assembly tasks. However, existing research primarily focuses on predicting limited human motions using static global information, which fails to address the highly stochastic nature of customized assembly operations in a given region. To address this, we propose a dynamic scenario-enhanced diverse human motion prediction network that extracts dynamic collaborative features to predict highly stochastic customized assembly operations. In this paper, we present a multi-level feature adaptation network that generates information for dynamically manipulating objects. This is accomplished by extracting multi-attribute features at different levels, including multi-channel gaze tracking, multi-scale object affordance detection, and multi-modal object’s 6 degree-of-freedom pose estimation. Notably, we employ gaze tracking to locate the collaborative space accurately. Furthermore, we introduce a multi-step feedback-refined diffusion sampling network specifically designed for predicting highly stochastic customized assembly operations. This network refines the outcomes of our proposed multi-weight diffusion sampling strategy to better align with the target distribution. Additionally, we develop a feedback regulatory mechanism that incorporates ground truth information in each prediction step to ensure the reliability of the results. Finally, the effectiveness of the proposed method was demonstrated through comparative experiments and validation of assembly tasks in a laboratory environment.