Variational Deep Clustering approaches for anomaly-based cyber-attack detection

Abstract

Detecting network anomalies is a critical cybersecurity task, yet existing methods struggle with high-dimensional data and limited interpretability in latent space. These challenges hinder precise differentiation between normal and anomalous activities due to (i) the chaotic distribution of normal samples, (ii) the absence of constraints to optimize the normal region’s hypervolume leading to high false alarm rates, (iii) the lack of prior knowledge for estimating the probability distribution of normal data, and (iv) slow inference times.

This research introduces two innovative deep generative models: Deep Clustering Variational Auto-Encoder (DCVAE) and Deep Clustering Support Vector Data Description Variational Auto-Encoder (DC-SVDD-VAE), designed to enhance learning latent features for detecting network anomalies. Both models incorporate a clustering layer within the Encoder to discover a clustering architecture suitable for normal network data. They also leverage prior information, specifically a Gaussian probability distribution, to estimate the posterior distribution that generates normal network data. Additionally, the DC-SVDD-VAE model integrates SVDD layers, which refine the clustering structure by mapping it onto an optimally sized hypersphere before computing the posterior probability. These approaches improve the separation between normal and abnormal regions at latent space, making it easier to identify significant/distinguishing latent features.

Both models were evaluated in conjunction with seven distinct one-class anomaly detection methods to assess the efficiency of the proposed solutions and the robustness of the generated features. These detectors were assessed using well-known intrusion benchmark datasets, including NSL-KDD, UNSW-NB15, CIC-IDS-2017, CSE-CIC-IDS-2018, and CTU-13. The experimental findings revealed that both models outperformed existing baselines and state-of-the-art approaches in terms of accuracy. Furthermore, inference stage processing times showed a notable decrease.