Autonomous Peer-to-Peer Energy Trading in Networked Microgrids: A Distributed Deep Reinforcement Learning Approach

Abstract

The emergence of networked microgrids in dereg-ulated electricity markets has transformed energy systems into multi-agent systems. In these systems, microgrids can proactively determine their policies and make intelligent decisions based on their own objectives. However, the current centralized energy market presents challenges for the facilitation of direct peerto-peer trading between microgrids, potentially restricting their ability to act proactively and make informed decisions. To address this issue, we propose a decentralized architecture to enable autonomous peer-to-peer trading in networked microgrids. This approach involves the modeling of peer-to-peer energy trading among microgrids as a partially observable Markov game, which is then solved using distributed deep reinforcement learning. Asynchronous proximal policy optimization, a specific algorithm of distributed deep reinforcement learning, is employed in a distributed setting to allow microgrids to effectively navigate the complex and competitive environment of peer-to-peer trading. peer-to-peer trading is also integrated with the physical network through the use of grid sensitivity analysis, which analyzes network constraints and transmission losses incurred during peerto-peer transactions. The proposed decentralized approach is evaluated using the CIGRE distribution network as a simulation environment, with the results demonstrating its effectiveness in facilitating market clearing for peer-to-peer energy trading.