A dynamic programming based method for optimal control of a cascaded heat pump system with thermal energy storage

Abstract The residential heating and cooling sector has been increasingly electrifying, predominantly using electrically driven heat pumps (HP) in combination with thermal/electrical energy storage systems. While these developments contribute to increased renewable and low carbon energy shares in the sector, exploiting the full potential of the technology requires a smart control of these systems that can account for predicted renewable energy availability in the future and the corresponding HP system performance. However, modelling a system featuring complex internal dynamics, in a way that is suitable for smart control, is challenging. Models need to be sophisticated enough to accurately capture the system's nonlinearities and intricacies while at the same time fast enough to enable a thorough search of the solutions space, in suitable computational time. Dynamic programming (DP) is a promising approach to smart controls, as it combines the ability to use complex, non-linear models while being an exhaustive search algorithm, guaranteeing that the global optimum is found. This paper presents an innovative modelling framework that entails reduced order models (ROM) of an HP substation's main components (i.e., HP and thermal energy storage—TES), elaborated in a fashion suitable for use in DP; these have been shaped as to include significant physical operating constraints (e.g., HP compressor variable speed, non-linear coefficient of performance—COP—dependency on outdoor and distribution temperature) affecting the system performance, while at the same time minimising the amount of state variables (i.e., TES temperatures, HP thermal and electric capacity) the optimizer needs to handle. In an application to an exemplary HP system, our system models compare remarkably well to detailed TRNSYS counterparts, used as a reference ground truth. The system achieves significant cost-saving enabled by the dynamic programming optimization approach, facilitating a 13% decrease in power consumption compared to conventional rule-based control.