DTKD-IDS: A dual-teacher knowledge distillation intrusion detection model for the industrial internet of things

Abstract

While advances in technology have brought great opportunities for the development of the Industrial Internet of Things (IIoT), cybersecurity risks are also increasing. Intrusion detection is a key technology to ensure the security and smooth operation of the Internet of Things (IoT), but owing to the resource constraints of IIoT devices, intrusion detection solutions need to be targeted and customized for the IIoT. This paper proposes a dual teacher knowledge distillation intrusion detection model called DTKD-IDS, which improves the performance of anomaly detection, accelerates the detection speed of the model, and reduces the complexity of the model. Specifically, To make the distillation process more efficient and stable, DTKD-IDS outputs a data prototype vector after each convolutional layer of the student network and the first teacher network. On the basis of these two prototype vectors, the student model extracts the most valuable knowledge from the structurally similar first teacher model. We name this process prototype distillation. In addition, we weight the extracted knowledge on the basis of the final classification loss of the two teacher networks and adaptively adjust the weights of the two teacher knowledge extractions during the training process to provide more accurate output distributions to guide the student network. This process is referred to as complementary distillation. During the training phase, we design a stable loss function to improve training efficiency. Through knowledge distillation, the model size and the number of parameters decreased by about 250 and 20 times compared to the first model, and by about 30 and 4 times compared to the second teacher model, while maintaining high detection performance. Numerous experimental results have shown that on the X-IIoTID, NSL-KDD and CICDDoS2019 datasets, the performance indicators of DTKD-IDS are improved compared with the traditional deep learning methods and the latest first-class models.