Reward function design in reinforcement learning for HVAC Control: A review of thermal comfort and energy efficiency Trade-offs

Reinforcement learning is increasingly applied to Heating, Ventilation, and Air Conditioning control to balance energy efficiency with occupant comfort. However, the design of the reward function, crucial for managing this trade-off, remains a relatively underexplored topic in existing review literature.

This paper addresses this gap through a systematic review of 79 studies published since 2020. We introduce a novel standardization methodology to enable a systematic comparison of the diverse reward formulations reported in the literature.

The analysis reveals two primary findings: (1) a substantial heterogeneity in reward function structures, with 68 unique designs identified, a factor that severely impedes research comparability; and (2) a prevalent reliance on empirically derived weighting coefficients that often lack a clear theoretical basis. Furthermore, this study identifies and quantifies the usage patterns of four common design techniques: occupancy consideration, comfort deadbands, error exponentiation, and acceptable limits.

Based on this comprehensive analysis, we propose a typical piecewise reward function structure that synthesizes common best practices and is grounded in established Heating, Ventilation, and Air Conditioning domain knowledge. This proposed structure is intended to serve as a foundational baseline, addressing the identified limitations and aiming to improve the comparability of future research in Reinforcement learning driven Heating, Ventilation, and Air Conditioning control.