A deep reinforcement learning driven bi-population evolutionary optimization for precast concrete scheduling with production transportation

Abstract

Precast concrete (PC) scheduling in prefabricated component production is essential for prefabricated building construction, where the transportation between different scaled factories and construction sites cannot be neglected. Additionally, multi-functional machines in factories can achieve flexible scheduling, enabling more efficient production for construction demand. Therefore, this study designs a bi-population based deep Q-network (B-DQN) with two cooperation populations to address the distributed flexible job shop scheduling problem with production transportation (DFJSPT) under PC manufacturing environment. Two objectives, i.e., makespan and total energy consumption, are minimized simultaneously. Firstly, in solution initialization, seven strategies concerning factory distribution, operation sequence, and machine assignment are built. Then, two deep Q-networks with 23 state features and 12 actions are designed to obtain better solutions, where DQN-G and DQN-L networks are to select global and local actions in global and local populations, respectively. In global and local actions, problem-specific and random heuristics are arranged to balance both exploitation and exploration of the B-DQN. Finally, dynamic switching mechanism enables cross-population solution migration to maintain evolutionary diversity. The comparison experiments with other competitive algorithms validates the effectiveness of the proposed approach in solving DFJSPT.