Transformer-based knowledge distillation for explainable intrusion detection system

Abstract

The rapid expansion of IoT networks has increased the risk of cyber threats, making intrusion detection systems (IDS) critical for maintaining security. However, most of the existing IDS rely on computationally intensive deep learning architectures, rendering them unsuitable for IoT environments with limited resources. Additionally, existing IDS approaches, including those using Knowledge Distillation (KD), often fail to capture the complex temporal dependencies and contextual relationships inherent in IoT traffic, which limits their ability to detect complex multi-stage attacks. Furthermore, these models frequently lack transparency, hindering effective decision-making by security experts. To address these gaps, we propose DistillGuard, a novel IDS framework designed specifically for IoT networks. The proposed framework employs a Transformer-based teacher model, which utilizes a hybrid attention mechanism combining multi-head self-attention (MHSA) and cross-attention layers to effectively capture both temporal and contextual patterns in network traffic. The framework further incorporates a Selective Gradient-Based Knowledge Distillation (SG-KD) process to transfer critical knowledge from the teacher model to a lightweight student model, optimizing performance while reducing computational costs. In addition, ‘DistillGuard’ integrates gradient contribution heatmaps, layer-wise contribution, and gradient selection impact analysis to provide detailed explanability, enabling security experts to understand which layers contribute to the detection of attacks. Experimental results demonstrate that ‘DistillGuard’ achieves superior detection accuracy and efficiency compared to existing state-of-the-art IDS models.