Improved Q-learning algorithm for solving permutation flow shop scheduling problems

Abstract

Generally, scheduling problems refer to allocation of available shared resources and the sorting of production tasks, in order to satisfy the specified performance target within a certain time. The fundamental scheduling problem is that all jobs need to be processed on the same route, which is called flow shop scheduling problems (FSSP). The goal of FSSP, proven as an NP-hard problem, is to find a job sequence that minimizes the makespan. In this paper, an improved Q-learning algorithm is proposed for solving the FSSP. Firstly, a problem model based on the basic Q-learning algorithm is constructed. The makespan is used as the feedback signal, and the process of environmental state change is defined as the process of job selection. Q-learning gives the expected utility of taking a given action in a given state. Afterwards, combined with the NEH heuristic, the algorithm efficiency is enhanced by changing the job inserting mode. In order to validate the proposed method, several simulation experiments are carried out on a set of test problems having different scales. The obtained optimization results of the proposed algorithm are compared to the standard Q-learning algorithm and a hybrid algorithm. The discussion and analysis show that the proposed algorithm performs better than the others in solving the permutation FSSP. As a future direction, in order to shorten the running time, further improvements will be studied to increase the performance of the proposed algorithm and make it applicable and efficient for solving multi-objective optimization problems.