A proximal policy optimization driven hyper-heuristic for workers constrained hybrid flow shop problem

As the production environment becomes increasingly complex, the integration of soft computing techniques becomes essential for addressing resource-constrained scheduling problems. This paper delves into a worker constrained hybrid flow shop scheduling problem (WHFSP) that integrates worker resources at each processing stage. A mixed-integer linear programming (MILP) model is constructed which enables the use of mathematical solvers to obtain optimal solutions for small-scale instances. Additionally, a novel soft computing-based scheduling framework, namely a proximal policy optimization-based hyper-heuristic algorithm (PPO-HH), is proposed. It automatically selects the most suitable low-level heuristic strategies based on the current state and historical data, facilitating efficient exploration and exploitation of the complex decision space. Several low-level heuristics including perturbative and local search operators are developed to explore the solution space. Subsequently, a high-level control strategy based on proximal policy optimization is proposed. A solution quality evaluation function and a reward mechanism based on problem characteristics are formulated. This mechanism provides feedback to PPO-HH based on the degree of alignment between the actions taken by the agent and the objectives, gradually optimizing the selection of low-level heuristic strategies. Eventually, it generates a probability distribution for each low-level heuristic in the given environment. Comprehensive numerical experiments are conducted to evaluate the performance of both the MILP model and the components of the PPO-HH algorithm. The comparison results show that PPO-HH is effective and efficient for solving the WHFSP.