Agent-based Modeling and Reinforcement Learning for Optimizing Energy Systems Operation and Maintenance: The Pathmind Solution

Abstract

The optimization of the Operation and Maintenance (O&M) of energy systems equipped with Prognostics and Health Management (PHM) capabilities can be framed as a sequential decision process, which can be addressed by Reinforcement Learning (RL). However, using RL algorithms requires specific skills, whereas the understanding of the possibly counter-intuitive solutions proposed by RL is not straifhtforward. To sidestep both issues, we use Pathmind, a software tool which enables effectively exploiting the RL capabilities without deep knowledge of machine learning. Pathmind is encoded in the Anylogic environment, which is an Agent-Based simulation software that simplifies the system modeling and allows easily visualizing the effects of the optimized policy. A scaled-down wind farm case study is used to demonstrate the potential of RL in identifying an optimal O&M policy and to show the ease of use of Pathmind and AnyLogic.