Digital-twin-based AGV cluster dynamic scheduling for solar cell production workshop using deep reinforcement learning

Abstract

In recent years, the demand for renewable energy sources, notably solar energy, has rapidly increased. As the most essential photovoltaic module, solar cells with high cleanliness and fragility rely on automated guided vehicles (AGVs) for transportation between various processes. However, the solar cell production workshop with massive AGVs has the characteristics of high dynamics, complexity, and uncertainty, which makes the traditional AGV scheduling methods unable to meet the dynamic scheduling requirements. Therefore, this paper proposes a digital-twin-based (DT-based) AGV cluster dynamic scheduling method using deep reinforcement learning (DRL). Firstly, a DT-based AGV cluster dynamic scheduling framework is constructed, ensuring operational synergy among DT, decision-making model formulation, and real-world application. Secondly, an AGV cluster dynamic scheduling mathematical model that minimizes the average waiting time is established. Thirdly, the problem of AGV cluster dynamic scheduling is transformed into a Markov Decision Process (MDP) with detailed descriptions. Moreover, an improved soft actor-critic (ISAC) DRL algorithm, adding the Softmax function to the actor network and introducing a multi-stage sample selection strategy, is implemented to resolve the established MDP model. Finally, the six cases derived from real-world solar cell production workshops are studied, and the results demonstrate the effectiveness of the proposed methodology.