Towards Adaptive Anomaly Detection in Buildings with Deep Reinforcement Learning

In this paper, we present early results on the use of deep reinforcement learning (DRL) for maximizing anomaly detection performance in buildings. We conjecture that DRL can improve performance by exploring the entire parameter space for all sensors, individually. Many anomaly detection algorithms are designed to use a single parameter for ease of use, however there are usually many parameter values that are pre-set, a priori. We hypothesize that a single threshold cannot work well for all sensors and propose the use of DRL to explore the entire parameter space. We use a deterministic policy gradient algorithm - Deep Deterministic Policy Gradient (DDPG)[4] - and use a building-specific anomaly detection algorithm, Strip, Bind, and Search (SBS) [2]. We find that while the maximum performance achieved by both approaches is similar, the DRL-based approach is significantly less biased, more consistent - up to 3x smaller standard deviation across individual sensor scores.