Anomaly detection in industrial control networks based on deep reinforcement learning and data reconstruction

Abstract

Any anomaly in the data flow of industrial control networks can directly translate into security vulnerabilities within the control system. Therefore, maintaining continuous situational awareness of industrial control networks and achieving precise identification and effective handling of abnormal data are critical to ensuring the reliability and functional safety of control systems. Anomaly detection tasks in industrial control networks aim to identify behavior deviating from normal patterns within network traffic or device data. Although existing unsupervised and self-supervised learning methods can train models without labeled data, their performance is limited by prior assumptions about data distributions, and it is difficult to ensure model stability when the training data is contaminated. To this end, this paper designs a Data Reconstruction and Anomaly Detection method, termed as DRAD. Specifically, the proposed method reconstructs the original data from the perspective of complete data-sample features, rather than relying solely on partial attribute learning. DRAD comprises four core modules. The state representation module maps industrial data stream samples to states of a Markov decision process. The action generation module outputs reconstructed data via a deterministic policy network. The state sampling module uniformly samples interactions from the simulated environment within the training set. The reward computation module designs instantaneous rewards based on reconstruction errors. Each module collaboratively optimizes within a deep deterministic policy gradient framework to maximize long-term cumulative rewards and learn normal data patterns. Moreover, DRAD gradually converges to a more robust detection strategy through long-term exploration, thereby significantly enhancing the stability of detection performance. Extensive experiments on eight datasets show that DRAD improves the AUC-ROC and AUC-PR by 6.8%-17.0% and 12.1%-47.3% and improves robustness under different anomaly contamination rates compared with seven representative competing methods.