Multi-agent systems in networked microgrids: Reinforcement learning and strategic pricing mechanisms

Abstract

This study presents a novel, multi-layered approach for optimizing power transactions in networked microgrids using a multi-agent system framework. It incorporates peer-to-peer trading within microgrid clusters and a market based mechanism for inter-microgrid cluster transactions. To facilitate trading efficiency and market coordination, microgrids are first grouped into clusters based on similar load profiles and generation characteristics, enabling efficient intra-microgrid cluster energy balancing before engaging in inter-microgrid cluster trading. Within each microgrid cluster, microgrids operate autonomously, using local optimization to assess power surpluses and shortages, followed by multi-agent reinforcement learning to dynamically determine bid/ask prices. The proposed framework integrates a two-tiered trading mechanism. First, intra-microgrid cluster trading is facilitated through a proportional bargaining pricing model, ensuring fair power distribution among microgrids within the same microgrid cluster. Then, inter-microgrid cluster trading is optimized using a system marginal pricing mechanism, allowing microgrid clusters to efficiently sell surplus and buy shortage while minimizing grid dependency. Simulations using real-world data demonstrate substantial cost reductions and improved market efficiency. The proposed approach achieves a reduction of 43.9% in the annual surplus energy sold to the grid which reduces reliance on the utility grid by 7.1%. Additionally, annual electricity purchase costs from the grid and the cost of selling electricity to the grid are decreased by 7.5% and 44.6%, respectively. These improvements contribute to greater energy self-sufficiency, lower transaction costs, and enhanced economic fairness among microgrids. This framework provides a scalable, effective, and market-driven solution for power trading in networked microgrids by integrating microgrid clustering, local optimization, dynamic bid/ask price learning, and decentralized trading mechanisms. This improves operational resilience and economic viability of future distributed power markets.