Multi-objective energy-efficient scheduling of distributed heterogeneous hybrid flow shops via multi-agent double deep Q-Network

Abstract

The Distributed Heterogeneous Hybrid Flow Shop Scheduling Problem (DHHFSP) poses a highly complex NP-hard combinatorial optimization challenge, particularly under the dual pressures of green manufacturing and distributed production. To address the limitations of conventional optimization algorithms in handling large-scale multi-objective scheduling with dynamic constraints, this paper proposes a novel Multi-Agent Double Deep Q-Network (MADDQN) based energy-efficient scheduling framework. The DHHFSP is formulated as a Markov Decision Process (MDP), where hierarchical agents representing jobs, workshops, and machines collaboratively learn optimal scheduling policies through a shared state representation and discrete rule-based action spaces. A hybrid reward mechanism combining delayed immediate rewards and global rewards is designed to efficiently guide the agents toward minimizing due time error (DTE) and total energy consumption (TEC). In addition, an adaptive energy-saving strategy is introduced to further reduce standby energy consumption without compromising delivery deadlines. Extensive computational experiments demonstrate that the proposed MADDQN achieves superior performance over state-of-the-art algorithms such as NSGA-II and optimal single rule methods in terms of convergence, solution diversity, and computational efficiency, with average improvements of 59.76%, 67.25%, and 99.72%, respectively. Furthermore, Pareto-based multi-objective evaluation metrics are utilized to comprehensively assess the balance between conflicting objectives. An industrial case study validates the practical applicability of the proposed method within real-world manufacturing execution systems (MES), offering a scalable and intelligent solution for energy-efficient scheduling in distributed heterogeneous manufacturing environments.