Smart residential electric vehicle charging and discharging scheduling via multi-agent asynchronous-updating deep reinforcement learning

Despite the increasing penetration of electric vehicle (EV), significant efforts are still required to transition towards a low-carbon future while balancing economy and power system stability. Coordinating EV charging and discharging scheduling in residential areas faces challenges due to uncertainties in EV owners’ commuting behaviors, complex energy demands, and unpredictable power information. This paper formulates the Residential EV Charging and Discharging Scheduling (REV-CDS) as a Markov Decision Process with an unknown transition function. Multi-Agent-Asynchronous-Soft-Actor-Critic (MAASAC) algorithm is proposed to solve the Markov Decision Process. The proposed method employs the asynchronous updating process instead of synchronous updating, which allows agents to maintain consistent policy update direction, enabling better learning of a charging and discharging strategy to improve coordination among EVs and highly align with the overall scheduling goals in the REV-CDS environment. Finally, several numerical studies were conducted to compare the proposed approach with classical multi-agent reinforcement learning methods. The studies demonstrate the effectiveness of improvement in minimizing charging costs, reducing carbon emissions, alleviating charging anxiety, and preventing transformer overload.