A systematic evaluation of white-box explainable AI methods for anomaly detection in IoT systems

Abstract

The rapid evolution of Internet of Things (IoT) systems has created new avenues for studying how artificial intelligence (AI) might be used to improve anomaly detection in these systems. Using AI for finding and predicting abnormalities or malfunctions in IoT devices with precision is crucial to highlight the salient characteristics of these anomalies to save maintenance expenses and downtime. Pairing these AI-based anomaly detection methods with explanations will make them applicable in real-world IoT systems. However, the systematic evaluation of various explainable AI (XAI) strategies used for anomaly detection in IoT systems is still lacking. To help address this gap, we propose an evaluation framework for different white-box XAI techniques in which we consider six evaluation metrics that are essential for evaluating XAI methods and have mappings to AI and security domains. In particular, we use these metrics to evaluate Integrated Gradients (IG), Layer-wise Relevance Propagation (LRP), and Deep Learning - SHAP (Deep-SHAP), through a single deep neural network (DNN) model. These metrics are descriptive accuracy, sparsity, stability, efficiency, robustness, and completeness. We used these six metrics to evaluate the three aforementioned white-box XAI methods considered in our work. Our evaluation framework is tested on two different IoT datasets: the N-BaIoT dataset, which focuses on detecting IoT botnet attacks on commercial IoT devices infected by the Mirai and Gafgyt botnets, and the MEMS dataset, which contains sensor data from manufacturing processes aimed at detecting abnormal readings. In the context of IoT systems, we illustrate the advantages and disadvantages of the state-of-the-art white-box XAI techniques through this study. Our evaluation framework’s source code is made accessible to the general public, providing a common benchmark for upcoming studies that explore applying explainable AI for enhancing anomaly detection in the emerging IoT field.