Multiagent Energy Management System Design Using Reinforcement Learning: The New Energy Lab Training Set Case Study

Abstract

This paper proposes a multiagent reinforcement learning (MARL) approach to optimize energy management in a grid-connected microgrid (MG). Renewable energy resources (RES) and customers are modeled as autonomous agents using reinforcement learning (RL) to interact with their environment. Agents are unaware of the actions or presence of others, which ensures privacy. Each agent aims to maximize its expected rewards individually. A double auction (DA) algorithm determines the price of the internal market. After market clearing, any unmet loads or excess energy are exchanged with the main grid. The New Energy Lab (NEL) at Staffordshire University is used as a case study, including wind turbines (WTs), photovoltaic (PV) panels, a fuel cell (FC), a battery, and various loads. We introduce a model-free Q-learning (QL) algorithm for managing energy in the NEL. Agents explore the environment, evaluate state-action pairs, and operate in a decentralized manner during training and implementation. The algorithm selects actions that maximize long-term value. To fairly consider the algorithms for both customers and producers, a fairness factor criterion is used. QL achieves a fairness factor of 1.2643, compared to 1.2358 for MC. It also has a shorter training time of 1483 compared with 1879.74 for MC and requires less memory, making it more efficient.