Physics-informed modularized neural network for advanced building control by deep reinforcement learning

Physics-informed machine learning (PIML) provides a promising solution for building energy modeling and can be used as a virtual environment to enable reinforcement learning (RL) agents to interact and learn. However, how to integrate physics priors efficiently, evaluate the effectiveness of physics constraints, balance model accuracy and physics consistency, and enable real-world implementation remain open challenges. To address these gaps, this study introduces a Physics-Informed Modularized Neural Network (PI-ModNN), which integrates physics priors through a physics-informed model structure, loss functions, and hard constraints. A new evaluation matrix called “temperature response violation” is developed to quantify the physical consistency of data-driven building dynamic models under varying control inputs and training data sizes. Additionally, a physics prior evaluation framework based on “rule importance” is proposed to quantify the contribution of each individual physical priors, offering guidance on selecting appropriate PIML techniques. The results indicate that incorporating physical priors does not always improve model performance; inappropriate physical priors could decrease model accuracy and consistency. However, hard constraints effectively enforce model consistency. Furthermore, we present a general workflow for developing control-oriented PIML models and integrating them with deep reinforcement learning (DRL). Following this framework, a case study of implementation DRL in an office space for three months demonstrates potential energy savings of 31.4%. Finally, we provide a general guideline for integrating data-driven models with advanced building control through a four-step evaluation framework, paving the way for reliable and scalable implementation of advanced building controls.