Selection hyperheuristic with knowledge-based Q-learning for dynamic distributed hybrid flow shop scheduling problem considering operation inspection

Abstract

In practical production environments, operation inspection plays a critical role in rescheduling defective products within the flow line, ensuring the smooth progression of subsequent processing stages. Despite its importance, this topic has received relatively little research attention. This paper addresses the dynamic distributed hybrid flow shop scheduling problem considering operation inspection (DHFSPI) aimed at minimizing makespan, where a operation of the job can either be scrapped or require reprocessing. A mathematical model is formulated for DHFSPI, and a selection hyperheuristic with knowledge-based Q-learning (SHKQL) is proposed to solve the problem. In SHKQL, eight pre-designed low-level heuristics (LLHs) are employed alongside knowledge-based Q-learning, which serves as the high-level heuristic (HLH). It adaptively selects these LLHs based on historical optimization knowledge. An initialization method is developed to construct the initial population, factoring in factory workload balance and random operation inspection. During the Q-learning process, a time-adaptive $ϵ$-greedy strategy is applied to guide the learning and application of historical knowledge. A rescheduling strategy is developed to address reprocessing and scrapping outcomes during operation inspection, considering production-specific characteristics. Benchmark instances of DHFSPI are constructed to evaluate the performance of SHKQL. The SHKQL is compared with several closely relevant scheduling methods through extensive experiments, and the results highlight its superior performance. This research provides valuable insights for managers dealing with dynamic distributed flow shop manufacturing systems, particularly those involving reprocessing and scrapping.