Digital twin-augmented proximal policy optimization for real-time home energy management: addressing uncertainties in renewables, pricing, and loads

Abstract

The escalating global energy demand, combined with intermittent renewable energy sources and dynamic electricity pricing, highlights the critical requirement for intelligent home energy management systems to optimize consumption, minimize expenses, and promote sustainability. Conventional home energy management systems face difficulties in managing uncertainties in power generation, household loads, and tariff variations, resulting in operational inefficiencies and increased grid pressure. This paper introduces a novel framework that integrates a digital twin with Proximal Policy Optimization for real-time home energy management, building on previous efforts by adding a digital twin for predictive validation and adaptive control. The framework structures the home energy management system as a Markov decision process, featuring extensive state-action spaces that handle energy storage systems, electric vehicles, photovoltaic panels, wind turbines, heating ventilation air conditioning, water heater, and schedulable loads through clipped Proximal Policy Optimization and generalized advantage estimation for reliable learning. Trained on a 180-day dataset and validated on a 96-step real-time dataset, it delivers 42.8% cost reduction from 23.68 to 13.53 euros compared to the baseline, achieves complete comfort compliance at 100%, and operates with 125 ms inference time, exceeding benchmarks by 7–30% through digital twin-enhanced robustness. This advancement establishes a scalable, resilient home energy management system for prosumer environments.