A trajectory-based algorithm enhanced by Q-learning and cloud integration for hybrid flexible flowshop scheduling problem with sequence-dependent setup times: A case study

Abstract

Eliminating non-production times in scheduling systems has seized attention for decades. Since scheduling problems have discrete search spaces with complex constraints, metaheuristic algorithms are commonly used by a notable number of researchers and practitioners. Although these algorithms do not guarantee optimality, they offer notable opportunities. Moreover, employing machine learning methods in such algorithms draws significant attention due to their promising capabilities such as learning patterns out from data for autonomous decision-making. Accordingly, this study introduces an Iterated Greedy Search algorithm enhanced by Q-learning method. In this regard, a new state evaluation method so as to process inputs in an aggregated fashion is proposed first. Different modifications of this function are adopted by two distinct Q-learning mechanisms. Accordingly, the proposed method autonomously tunes both algorithm parameters and local search procedures. Secondarily, a cloud-integrated scheduler adopting the proposed method is developed as a prototype model. Thus, information derived out from data can be shared among devices and plants at any location. The proposed strategy is tested on a hybrid flexible flowshop scheduling problem with sequence-dependent setup times and release times, which has numerous applications in industry. The performance of the proposed approach is compared to a number of well-regarded and commonly used algorithms. In this context, synthetic problem data is used first. Subsequent to demonstration of the superiority of the proposed approach in these problems and conducting comparisons with CPLEX solver, it is tested on production data. Comprehensive experimental study and statistically verified results point out the efficiency of the proposed approach.