SKT-IDS: Unknown attack detection method based on Sigmoid Kernel Transformation and encoder–decoder architecture

Intrusion Detection Systems (IDS) are crucial in cybersecurity for monitoring network traffic and identifying potential attacks. Existing IDS research largely focuses on known attack detection, leaving a significant gap in research regarding unknown attack detection, where achieving a balance between false alarm rate (identifying normal traffic as attack traffic) and recall rate of unknown attack detection remains challenging. To address these gaps, we propose a novel IDS based on Sigmoid Kernel Transformation and Encoder-Decoder architecture, namely SKT-IDS, where SKT stands for Sigmoid Kernel Transformation. We start with pre-training an attention-based encoder for coarse-grained intrusion detection. Then, we use this encoder to build an encoder–decoder model specifically for 0-day attack detection, training it solely on known traffic using the cosine similarity loss function. To enhance detection, we introduce a Sigmoid Kernel Transformation for feature engineering, improving the discriminative ability between normal traffic and 0-day attacks. Finally, we conducted a series of ablation and comparative experiments on the NSL-KDD and CSE-CIC-IDS2018 datasets, confirming the effectiveness of our proposed method. With a false alarm rate of 1%, we achieved recall rates for unknown attack detection of 65% and 69% on the two datasets, respectively, demonstrating significant performance improvements compared to existing state-of-the-art models.