Modeling Participation of Storage Units in Electricity Markets using Multi-Agent Deep Reinforcement Learning

Abstract

Modeling electricity markets realistically plays a crucial role for understanding complex emerging market dynamics and guiding policy making. In systems with a high share of variable renewable generation, accurately modeling the behavior of storage units can be particularly challenging, as their bidding strategies depend on expected electricity prices. While there exist a wide variety of electricity market models, they typically rely on rule-based bidding strategies or optimization approaches, which may not be sufficient to represent competitive and strategic behavior on the market. In this paper, we present a multi-agent deep reinforcement learning modeling framework that allows representing competitive and strategic behavior of energy storage units. This framework can be executed in large-scale electricity market models, thus facilitating market design analyses. We show that the proposed approach performs very well when compared with widely used modeling approaches, and its computational efficiency makes its use in energy market modeling practical.