A multi-step and resilient predictive Q-learning algorithm for IoT: a case study in water supply networks

Abstract

In this paper, we consider the problem of deriving recommended resilient and predictive actions for an IoT network in the presence of faulty components and malicious agents. The IoT, combining physical and cyber devices, is formulated as a directed graph with a known topology whose objective is to maintain a constant and resilient flow between a source node and a destination node. The optimal route through this network is evaluated via a predictive and resilient Q-learning algorithm which takes into account historical data about irregular operation, including faults and attacks. To showcase the efficacy of our approach, we utilize anonymized data from Arlington County, Virginia to obtain predictive and resilient scheduling policies for a smart water supply system while avoiding neighborhoods with leaks and other faults.