Automated reinforcement learning for sequential ordering problem using hyperparameter optimization and metalearning

Abstract

AutoML systems seek to assist Artificial Intelligence users in finding the best configurations for machine learning models. Following this line, recently the area of Automated Reinforcement Learning (AutoRL) has become increasingly relevant, given the growing increase in applications for reinforcement learning algorithms. However, the literature still lacks specific AutoRL systems for combinatorial optimization, especially for the Sequential Ordering Problem (SOP). Therefore, this paper aims to present a new AutoRL approach for SOP. For this, two new methods are proposed using hyperparameter optimization and metalearning: AutoRL-SOP and AutoRL-SOP-MtL. The proposed AutoRL techniques enable the combined tuning of three SARSA hyperparameters, being ϵ-greedy policy, learning rate, and discount factor. Furthermore, the new metalearning approach enables the transfer of hyperparameters between two combinatorial optimization domains: TSP (source) and SOP (target). The results show that the application of metalearning generates a reduction in computational cost in hyperparameter optimization. Furthermore, the proposed AutoRL methods achieved the best solutions in 23 out of 28 simulated TSPLIB instances compared to recent literature studies.