{"title":"OCTOPUS: Deep Reinforcement Learning for Holistic Smart Building Control","authors":"Xianzhong Ding, Wan Du, Alberto Cerpa","doi":"10.1145/3360322.3360857","DOIUrl":"https://doi.org/10.1145/3360322.3360857","url":null,"abstract":"Recently, significant efforts have been done to improve quality of comfort for commercial buildings' users while also trying to reduce energy use and costs. Most of these efforts have concentrated in energy efficient control of the HVAC (Heating, Ventilation, and Air conditioning) system, which is usually the core system in charge of controlling buildings' conditioning and ventilation. However, in practice, HVAC systems alone cannot control every aspect of conditioning and comfort that affects buildings' occupants. Modern lighting, blind and window systems, usually considered as independent systems, when present, can significantly affect building energy use, and perhaps more importantly, user comfort in terms of thermal, air quality and illumination conditions. For example, it has been shown that a blind system can provide 12%~35% reduction in cooling load in summer while also improving visual comfort. In this paper, we take a holistic approach to deal with the trade-offs between energy use and comfort in commercial buildings. We developed a system called OCTOPUS, which employs a novel deep reinforcement learning (DRL) framework that uses a data-driven approach to find the optimal control sequences of all building's subsystems, including HVAC, lighting, blind and window systems. The DRL architecture includes a novel reward function that allows the framework to explore the trade-offs between energy use and users' comfort, while at the same time enable the solution of the high-dimensional control problem due to the interactions of four different building subsystems. In order to cope with OCTOPUS's data training requirements, we argue that calibrated simulations that match the target building operational points are the vehicle to generate enough data to be able to train our DRL framework to find the control solution for the target building. In our work, we trained OCTOPUS with 10-year weather data and a building model that is implemented in the EnergyPlus building simulator, which was calibrated using data from a real production building. Through extensive simulations we demonstrate that OCTOPUS can achieve 14.26% and 8.1% energy savings compared with the state-of-the art rule-based method in a LEED Gold Certified building and the latest DRL-based method available in the literature respectively, while maintaining human comfort within a desired range.","PeriodicalId":128826,"journal":{"name":"Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126094260","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Heterogeneous Transfer Learning for Thermal Comfort Modeling","authors":"W. Hu, Yong Luo, Zongqing Lu, Yonggang Wen","doi":"10.1145/3360322.3360843","DOIUrl":"https://doi.org/10.1145/3360322.3360843","url":null,"abstract":"For decades, the Predicted Mean Vote (PMV) model has been adopted to evaluate building occupants' thermal comfort. However, recent studies argue that the PMV model is inaccurate and suffers from two major issues: thermal comfort parameter inadequacy and modeling data inadequacy. To overcome these issues, in this paper, we propose a learning-based approach for thermal comfort modeling, named as Heterogeneous Transfer Learning (HTL) based Intelligent Thermal Comfort Neural Network (HTL-ITCNN). First, to address the parameter inadequacy issue, we add more relevant factors as the modeling features except for the six PMV parameters. Due to the flexibility of learning-based approaches, newly found thermal comfort parameters can be appended to extend the number of modeling features. Second, to mitigate the impact of the data inadequacy issue, we adopt the deep transfer learning techniques to train the thermal comfort model, where the model training would benefit from the transferred knowledge from the existing datasets. Due to the heterogeneity of the features among different datasets, we follow the HTL concept to conducting effective knowledge transfer among heterogeneous domains, which are the different but related datasets with varied features. To validate our solution, we conduct five-month data collection experiments and build our datasets. With the HTL-based two-stage learning paradigm, the experimental results show that the accuracy of HTL-ITCNN outperforms the PMV model by on average 73.9%. Besides, we verify the impacts of newly added features and knowledge transfer on model performance. Moreover, we demonstrate the enormous potential of personal thermal comfort modeling research.","PeriodicalId":128826,"journal":{"name":"Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127327835","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Towards Adaptive Anomaly Detection in Buildings with Deep Reinforcement Learning","authors":"Tong Wu, Jorge Ortiz","doi":"10.1145/3360322.3361011","DOIUrl":"https://doi.org/10.1145/3360322.3361011","url":null,"abstract":"In this paper, we present early results on the use of deep reinforcement learning (DRL) for maximizing anomaly detection performance in buildings. We conjecture that DRL can improve performance by exploring the entire parameter space for all sensors, individually. Many anomaly detection algorithms are designed to use a single parameter for ease of use, however there are usually many parameter values that are pre-set, a priori. We hypothesize that a single threshold cannot work well for all sensors and propose the use of DRL to explore the entire parameter space. We use a deterministic policy gradient algorithm - Deep Deterministic Policy Gradient (DDPG)[4] - and use a building-specific anomaly detection algorithm, Strip, Bind, and Search (SBS) [2]. We find that while the maximum performance achieved by both approaches is similar, the DRL-based approach is significantly less biased, more consistent - up to 3x smaller standard deviation across individual sensor scores.","PeriodicalId":128826,"journal":{"name":"Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation","volume":"35 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114812253","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Skin Temperature Extraction Using Facial Landmark Detection and Thermal Imaging for Comfort Assessment","authors":"Ashrant Aryal, B. Becerik-Gerber","doi":"10.1145/3360322.3360848","DOIUrl":"https://doi.org/10.1145/3360322.3360848","url":null,"abstract":"Despite the large share of energy consumption, current HVAC systems in buildings fail to meet their primary purpose of maintaining comfortable indoor conditions. Current \"one size fits all\" approach to control the thermal conditions in an environment lead to a high degree of occupant dissatisfaction. Advancements in Internet of Things and Machine Learning have opened the possibility of deploying different sensors at a wide scale to monitor environmental and physiological information and using collected sensor data to model individual comfort requirements. Thermal imaging has recently gained interest as one of the possible ways to monitor physiological information (skin temperature) for thermal comfort assessment. Previous studies have shown that skin temperatures from different regions of the face, such as forehead, nose, cheeks and ears can provide useful information for predicting thermal sensation at an individual level. However, existing approaches to process thermal images either rely on manual temperature extraction or use methods that are less reliable in accurately identifying different facial regions. One of the major challenges of using thermal imaging for monitoring skin temperatures in actual buildings is that occupants may move relative to the camera. It is not practical to expect building occupants to be oriented facing the cameras at all times, therefore, it is important to be able to extract as much information as possible from instances where it is feasible to extract relevant information. In this paper, we describe an approach to extract skin temperature by locating specific regions of the face in thermal images. The approach involves combining data from RGB images with thermal images and leveraging facial landmark detection in RGB images. We also evaluate our approach with existing approach of face detection used in previous studies. Our study demonstrates that facial landmark detection provides a more accurate calculation of different locations in the face compared to previous studies. We show an improvement in overall quantity and quality of temperature measurements extracted from thermal images compared to previous studies. More accurate temperature measurements from thermal images can improve the accuracy of thermal imaging for modeling and predicting thermal comfort.","PeriodicalId":128826,"journal":{"name":"Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation","volume":"433 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131607183","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"No Time to Waste, with SWAM!","authors":"S. Faye, F. Melakessou, P. Gautier, D. Khadraoui","doi":"10.1145/3360322.3360991","DOIUrl":"https://doi.org/10.1145/3360322.3360991","url":null,"abstract":"Managing waste from professional customers (e.g. restaurants, shops) has an impressive number of requirements that must be met to ensure a high quality of service and environmental compliance. Existing decision support systems generally rely on a limited flow of information and offer an often static or statistically based approach. With SWAM, we aim to create a smart waste collection system relying on data generated by new fill-level sensor technologies integrated into waste bins. Through this demonstration, we intend to show how artificial intelligence and new sensing technologies can be of benefit to this sector. The project concept will be showcased through a small-scale interactive demonstration where the audience will be responsible for the proper execution of the waste collection processes. A video of the demo is available online1 -It shows the interaction between the audience, our decision-making systems, waste collection robot and miniature bins equipped with ultrasonic sensors.","PeriodicalId":128826,"journal":{"name":"Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123693959","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Xiaoxia Li, Wei Li, Qiang Yang, W. Yan, Albert Y. Zomaya
{"title":"Building an Online Defect Detection System for Large-scale Photovoltaic Plants","authors":"Xiaoxia Li, Wei Li, Qiang Yang, W. Yan, Albert Y. Zomaya","doi":"10.1145/3360322.3360835","DOIUrl":"https://doi.org/10.1145/3360322.3360835","url":null,"abstract":"The power efficiency of photovoltaic modules is highly correlated with their health status. Under dynamically changing environments, photovoltaic defects could spontaneously form and develop into fatal faults during the daily operation of photovoltaic plants. To facilitate defect detection with less human intervention, a nondestructive and contactless visual inspection system with the help of unmanned aerial vehicles and edge computing is proposed in this work. Limited by the resources of edge devices and the availability of images of photovoltaic defects for training, we developed an online solution combined with deep learning, data argumentation and transfer learning to properly address the issues of running resource hungry applications on edge devices and lack of training samples faced by the deep learning approaches used in the field. With the reduction of the network depth of the deep convolutional neural network model and the transfer of features from the learned defects, the resource consumption of our proposed approach is significantly reduced, and thus can be used on a wide range of edge devices to complete defect detection in a timely manner with high accuracy. To study the performance of our design, a testbed was built from open source hardware and software, and field trials were carried out in three photovoltaic plants. The experimental results clearly demonstrate the practicality and effectiveness of our design for detecting visible defects on photovoltaic modules.","PeriodicalId":128826,"journal":{"name":"Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122412857","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Nipun Batra, Rithwik Kukunuri, Ayush Pandey, Raktim Malakar, Rajat Kumar, Odysseas Krystalakos, Mingjun Zhong, Paulo C. M. Meira, Oliver Parson
{"title":"Towards reproducible state-of-the-art energy disaggregation","authors":"Nipun Batra, Rithwik Kukunuri, Ayush Pandey, Raktim Malakar, Rajat Kumar, Odysseas Krystalakos, Mingjun Zhong, Paulo C. M. Meira, Oliver Parson","doi":"10.1145/3360322.3360844","DOIUrl":"https://doi.org/10.1145/3360322.3360844","url":null,"abstract":"Non-intrusive load monitoring (NILM) or energy disaggregation is the task of separating the household energy measured at the aggregate level into constituent appliances. In 2014, the NILM toolkit (NILMTK) was introduced in an effort towards making NILM research reproducible. Despite serving as the reference library for data set parsers and reference benchmark algorithm implementations, few publications presenting algorithmic contributions within the field went on to contribute implementations back to the toolkit. This paper describes two significant contributions to the NILM community in an effort towards reproducible state-of-the-art research: i) a rewrite of the disaggregation API and a new experiment API which lower the barrier to entry for algorithm developers and simplify the definition of algorithm comparison experiments, and ii) the release of NILMTK-contrib; a new repository containing NILMTK-compatible implementations of 3 benchmarks and 9 recent disaggregation algorithms. We have performed an extensive empirical evaluation using a number of publicly available data sets across three important experiment scenarios to showcase the ease of performing reproducible research in NILMTK.","PeriodicalId":128826,"journal":{"name":"Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116809880","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
José R. Vázquez-Canteli, J. Kämpf, G. Henze, Z. Nagy
{"title":"CityLearn v1.0: An OpenAI Gym Environment for Demand Response with Deep Reinforcement Learning","authors":"José R. Vázquez-Canteli, J. Kämpf, G. Henze, Z. Nagy","doi":"10.1145/3360322.3360998","DOIUrl":"https://doi.org/10.1145/3360322.3360998","url":null,"abstract":"Demand response has the potential of reducing peaks of electricity demand by about 20% in the US, where buildings represent roughly 70% of the total electricity demand. Buildings are dynamic systems in constant change (i.e. occupants' behavior, refurbishment measures), which are costly to model and difficult to coordinate with other urban energy systems. Reinforcement learning is an adaptive control algorithm that can control these urban energy systems relying on historical and real-time data instead of models. Plenty of research has been conducted in the use of reinforcement learning for demand response applications in the last few years. However, most experiments are difficult to replicate, and the lack of standardization makes the performance of different algorithms difficult, if not impossible, to compare. In this demo, we introduce a new framework, CityLearn, based on the OpenAI Gym Environment, which will allow researchers to implement, share, replicate, and compare their implementations of reinforcement learning for demand response applications more easily. The framework is open source and modular, which allows researchers to modify and customize it, e.g., by adding additional storage, generation, or energy-consuming systems.","PeriodicalId":128826,"journal":{"name":"Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation","volume":"10 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131056430","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Gnu-RL: A Precocial Reinforcement Learning Solution for Building HVAC Control Using a Differentiable MPC Policy","authors":"Bingqing Chen, Zicheng Cai, M. Berges","doi":"10.1145/3360322.3360849","DOIUrl":"https://doi.org/10.1145/3360322.3360849","url":null,"abstract":"Reinforcement learning (RL) was first demonstrated to be a feasible approach to controlling heating, ventilation, and air conditioning (HVAC) systems more than a decade ago. However, there has been limited progress towards a practical and scalable RL solution for HVAC control. While one can train an RL agent in simulation, it is not cost-effective to create a model for each thermal zone or building. Likewise, existing RL agents generally take a long time to learn and are opaque to expert interrogation, making them unattractive for real-world deployment. To tackle these challenges, we propose Gnu-RL: a novel approach that enables practical deployment of RL for HVAC control and requires no prior information other than historical data from existing HVAC controllers. To achieve this, Gnu-RL adopts a recently-developed Differentiable Model Predictive Control (MPC) policy, which encodes domain knowledge on planning and system dynamics, making it both sample-efficient and interpretable. Prior to any interaction with the environment, a Gnu-RL agent is pre-trained on historical data using imitation learning, which enables it to match the behavior of the existing controller. Once it is put in charge of controlling the environment, the agent continues to improve its policy end-to-end, using a policy gradient algorithm. We evaluate Gnu-RL on both an EnergyPlus model and a real-world testbed. In both experiments, our agents were directly deployed in the environment after offline pre-training on expert demonstration. In the simulation experiment, our approach saved 6.6% energy compared to the best published RL result for the same environment, while maintaining a higher level of occupant comfort. Next, Gnu-RL was deployed to control the HVAC of a real-world conference room for a three-week period. Our results show that Gnu-RL saved 16.7% of cooling demand compared to the existing controller and tracked temperature set-point better.","PeriodicalId":128826,"journal":{"name":"Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation","volume":"90 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128826547","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"GridInSight","authors":"Zeal Shah, Alex Yen, Ajey Pandey, Jayadeva Tanejā","doi":"10.1145/3360322.3360855","DOIUrl":"https://doi.org/10.1145/3360322.3360855","url":null,"abstract":"We demonstrate GridInSight, a suite of techniques that leverage low-cost, non-intrusive, and commodity smartphone and machine vision cameras to measure electricity grids. Specifically, we develop techniques to measure electricity grid frequency, phase (indoors), and phase (outdoors) across a mix of cameras with errors of 1-2%, 2-5%, and 3-10%, respectively. Further, we develop a novel technique and show an error of 8-15% for measuring voltage on a lightbulb that our system had not seen previously. The ability to cheaply and pervasively measure power quality with non-intrusive, off-the-shelf hardware can enable a wide range of applications for monitoring electricity grids, particularly in emerging economies.","PeriodicalId":128826,"journal":{"name":"Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122517000","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}