Hierarchical decision and control method for the human–exoskeleton collaborative packaging system based on deep reinforcement learning

Abstract

The packaging process is an important part of the production and transportation process, and many companies have introduced exoskeleton robots to mitigate worker fatigue in the packaging process. Therefore, a human–exoskeleton collaborative packaging system with random product arrivals is studied, focusing on the operation decision and assistive force control of the system. A hierarchical decision and control policy is proposed to prevent the worker’s fatigue level from crossing a threshold while improving the energy efficiency of the exoskeleton and the system’s productivity. First, a hierarchical decision and control architecture is designed, in which the upper layer makes decisions on operations and the lower layer controls the assistive forces. Second, the optimal hierarchical decision and control policy is solved by combining the double DQN (DDQN) for discrete actions and the deep deterministic policy gradient (DDPG) for continuous actions. Finally, the proposed policy is validated in the constructed visualization virtual platform. The simulation results show that the proposed policy can effectively control worker fatigue and improve the energy efficiency of the equipment and the productivity of the system.