A Method to Evaluate the Performance of Predictors in Cyber-Physical Systems

Abstract

Cyber-Physical Systems (CPS) rely on sensing to control and optimize their operation. Nevertheless, sensing itself is prone to errors that can originate at several stages, from sampling to communication. In this context, several systems adopt multivariate predictors to assess the quality of the sensed data, to replace data from faulty sensors, or to derive variables that cannot be directly sensed. These predictors are often evaluated based on their accuracy and computing demands, however, such evaluations often do not consider the system's architecture from a broader perspective, ignoring the way components are interconnected and how they cascade as inputs of other Machine Learning (ML) models. In this work, we introduce a method to evaluate the performance of interdependent predictors based on the stability of the estimation error dynamics in faulty scenarios. The proposed method estimates the ability of a predictor to produce accurate predictions while accounting for the impacts of cascading predicted values as its inputs. The prediction correctness is estimated based solely on information acquired during the training of the multivariate predictors and mathematical properties of the ML activation functions. The proposed method is evaluated with a meaningful dataset in the scope of monitoring and control of a Cyber-Physical System, and the evaluation demonstrates the ability of the proposed method to account for the interdependence of data predictors.