An Energy Optimization Algorithm for Data Centers based on Deep Q-learning with Multi-Source Energy

More and more data centers are supplied by multi-source energy. However, the features of random, uncertain, and time-varying of renewable energy has made it difficult to achieve good results with traditional methods. In this paper, we research how to coordinate multiple energy sources (such as wind power, solar, and smart grids) to reduce energy costs of data centers. We propose a deep Q-learning (DQN) algorithm based on the auto encoder to control the energy consumption of data center. Our algorithm uses the auto encoder to approximate the Q-value function, learning the expected cost based on the state of current system. It solves the problem that the Q-value function in traditional Q-learning algorithm is difficultly designed under multi-constraint conditions, and it can converge by any state of the system to obtain the optimal solution. In order to further improve the convergence speed and accuracy of the algorithm. We design a parameter optimization strategy to solve the slow convergence problem of the algorithm. This strategy is based on the experience replay technology to optimize the parameters of algorithm. We conducted extensive experiments based on real- world data, and evaluated the performance of our algorithm. The experiment results show that our algorithm can average save 20% energy cost so as to bring a set of safe and highly available solution to meet the requirements of multi-Source energy for data centers.