A Q-learning-based multi-objective hyper-heuristic algorithm for energy-efficient integrated distributed hybrid flow-shop scheduling with preventive maintenance

Abstract

Driven by the dual engines of supply chain integration and low-carbon transformation in industrial Internet of Things (IIoT) manufacturing systems, energy-efficient integrated distributed scheduling has emerged as a pivotal component of industrial intelligence-driven smart manufacturing. This article investigates the energy-efficient integrated distributed hybrid flow shop scheduling problem with preventive maintenance (EE-IDHFSP-PM), which aims to minimize the dual objectives of makespan and total carbon emissions. In this study, a mixed-integer linear programming (MILP) model is established for the EE-IDHFSP-PM, making the first attempt to solve such NP-hard problem by using a Q-learning-based multi-objective hyper-heuristic algorithm (QLMHHA). First, a modified NEH-based initialization method is introduced to produce high-quality solutions that balance multiple optimization objectives, ensuring both the quality and diversity of initial populations. Second, a novel multi-stage collaborative energy-efficient strategy (MSC_EES) is developed to dynamically adjust the processing speeds of machines on non-critical paths, which reduces energy consumption across stages. Third, a new Q-learning-based high-level strategy (HLS) is devised to dynamically coordinate twelve low-level heuristics (LLHs) according to specific states, improving adaptive search efficiency through superior exploration–exploitation trade-offs. Fourth, a dual-criterion reward mechanism is proposed to evaluate population quality in terms of both convergence and diversity, which can deliver immediate feedback and effectively guide evolutionary processes. Fifth, comprehensive convergence and computational complexity analyses of critical components are conducted to confirm the stability, reliability, and efficiency of QLMHHA. Extensive experiments are carried out on 54 small-scale and 24 large-scale instances, which demonstrate that QLMHHA achieves promising performance in both effectiveness and efficacy against state-of-the-art multi-objective algorithms for addressing the EE-IDHFSP-PM. These findings validate the efficacy and superiority of QLMHHA in tackling complex scheduling challenges, providing valuable theoretical implications and practical insights for energy-efficient distributed manufacturing systems.