Energy-efficient and self-adaptive AGV scheduling approach based on hierarchical reinforcement learning for flexible shop floor

Driven by the recent trend of Industry 4.0, Automated Guided Vehicles (AGVs) have been widely applied in manufacturing industry to enhance the efficiency of the logistics system. However, the application of AGVs also aroused issues such as increasing energy consumption and various costs, especially in real-time AGVs scheduling in the complex flexible shop floor. To address these issues, a hierarchical reinforcement learning (HRL) based approach is hereby proposed to achieve real-time AGVs scheduling. At first, the scheduling task is decomposed into task assignment and AGV selection subtasks with the concept of hierarchy, and the problem of real-time AGVs scheduling is formulated as a Semi-Markov decision process (SMDP), aiming to simultaneously minimize makespan and total operational cost aroused by energy consumption, delay ratio, and maintenance. Then the HRL based real-time AGVs scheduling is presented to implement task assignment and AGV selection. In the end, a case study is illustrated to validate the effectiveness and superiority of the proposed approach. The results show that the maximum reduction of energy, maintenance, and total operational cost is 33.4%, 25.9%, and 24.1% respectively.