A DQN based approach for large-scale EVs charging scheduling

Abstract

This paper addresses the challenge of large-scale electric vehicle (EV) charging scheduling during peak demand periods, such as holidays or rush hours. The growing EV industry has highlighted the shortcomings of current scheduling plans, which struggle to manage surge large-scale charging demands effectively, thus posing challenges to the EV charging management system. Deep reinforcement learning, known for its effectiveness in solving complex decision-making problems, holds promise for addressing this issue. To this end, we formulate the problem as a Markov decision process (MDP). We propose a deep Q-learning (DQN) based algorithm to improve EV charging service quality as well as minimizing average queueing times for EVs and average idling times for charging devices (CDs). In our proposed methodology, we design two types of states to encompass global scheduling information, and two types of rewards to reflect scheduling performance. Based on this designing, we developed three modules: a fine-grained feature extraction module for effectively extracting state features, an improved noise-based exploration module for thorough exploration of the solution space, and a dueling block for enhancing Q value evaluation. To assess the effectiveness of our proposal, we conduct three case studies within a complex urban scenario featuring 34 charging stations and 899 scheduled EVs. The results of these experiments demonstrate the advantages of our proposal, showcasing its superiority in effectively locating superior solutions compared to current methods in the literature, as well as its efficiency in generating feasible charging scheduling plans for large-scale EVs. The code and data are available by accessing the hyperlink: https://github.com/paperscodeyouneed/A-Noisy-Dueling-Architecture-for-Large-Scale-EV-ChargingScheduling/tree/main/EV%20Charging%20Scheduling.