Residential Energy Management Method Based on the Proposed A3C-FER

Abstract

Deep reinforcement learning has been widely applied in the field of residential energy management, showcasing considerable promise in enhancing energy efficiency and reducing energy consumption. However, it is observed that some methodologies still suffer from inadequate data exploitation, which results in suboptimal policy performance. In this study, focusing on the residential energy management system, an innovative reinforcement learning method is proposed. This novel method fuses the asynchronous advantage actor-critic architecture with a familiarity-based experience replay mechanism, with the ambition of markedly improving learning efficiency and control performance. Numerical comparisons were made to justify the effectiveness of the method. Experimental results across diverse cases confirm that the proposed algorithm can effectively achieve optimal energy scheduling for residential sectors. Furthermore, the proposed methodology has demonstrated a notable reduction in grid interaction expenses, achieving a decrease of 27.03% and 16.38% relative to the other two scenarios. In comparison with the Proximal Policy Optimization (PPO) and Deep Q-Network (DQN) algorithms, the novel approach not only improves the average reward value post-convergence by 38.48% and 47.17%, respectively, but also significantly reduces the training duration by 81.19% within a multi-threaded computational environment.