Dueling-DDQN Based Virtual Machine Placement Algorithm for Cloud Computing Systems

Abstract

Virtual machine placement (VMP) in large-scale cloud computing clusters is a challenging problem with practical importance. Deep Q-learning (DQN) based algorithm is a promising means to solve difficult VMP problems with complex optimization goals and dynamically changing environments. However, native DQN algorithms suffer from shortcomings such as Q value overestimation, difficulty in convergence, and failure to maximize long-term reward. To overcome these shortcomings, this paper proposes an advanced VMP algorithm based on Dueling-DDQN. Moreover, specific optimization techniques are introduced to enhance the exploration strategy and the capability of achieving long-term reward. Experiment results show that the proposed algorithm outperforms native DQN in terms of convergence speed, Q-value estimation accuracy and stability. Meanwhile, the proposed algorithm can achieve multiple optimization goals such as reducing power consumption, ensuring resource load balance and Improving user service Quality.