Contrastive-Learning-Based Decision Making for Dynamic Time-Linkage Optimization

Abstract

In dynamic time-linkage optimization, current decisions influence the future state of environments. To make good decisions that have a positive impact on future states, existing methods usually build a model to predict the future rewards of solutions for decision making. However, these prediction models present low accuracy since decision data are not enough to train such a complex model. To address this issue, this article proposes a contrastive-learning-based decision making (CLDM) method, which builds a contrastive model to learn the relationship between solutions but not absolute rewards and adopts a quick decision strategy to select solutions. In CLDM, a clustering-based time-linkage detection (CD) strategy is developed to measure the intensity of the time linkage, which determines whether to make decisions based on future rewards. To represent the relative relationship between solutions, a large number of contrastive samples are constructed using the limited historical decisions. A contrastive model is trained for solution comparison in terms of the combination of current fitness and future rewards. Candidate solutions are clustered into multiple groups to filter poor ones, and a few solutions are preserved to rank using the contrastive model. The winner is taken as the decision solution. Integrating CLDM into particle swarm optimization (PSO), a new algorithm named contrastive-learning-based PSO (CL-PSO) is put forward. Experimental results on multiple dynamic time-linkage optimization instances demonstrate that CL-PSO outperforms state-of-the-art algorithms in terms of solution quality. CL-PSO can also well solve the mobile robot path planning problem.