Unsupervised CoAP-based anomaly detection in private 5G core network for Industrial-IoT

Abstract

In the ever-evolving Industry 4.0 landscape, logs and events play a critical role in maintaining the reliability and security of complex systems. This is especially true for 5G networks, where lightweight protocols such as CoAP and LwM2M produce massive volumes of structured but often unlabeled log data. Although machine learning and deep learning methods have become common tools for anomaly detection in such environments, they are frequently hindered by practical limitations: scarce labeled data, high-dimensionality, long log sequences, and the lack of realistic attack scenarios in available datasets. To address the above challenges, we adopt a deep clustering framework that learns low-dimensional structured latent representations without requiring labels. TF–IDF statistical analysis excels at detecting repetitive attack patterns and anomalous term frequencies characteristic of threats like distributed denial-of-service attacks and message flooding, while enabling rapid compression and interpretation of large-scale log data. However, TF–IDF alone fails to capture semantic context—for instance, distinguishing between benign timeouts and attack-induced timeout cascades. Conversely, while pre-trained models like sentence-transformer capture rich semantics, their latent spaces are optimized for classification and are often unsuitable for clustering, due to their entangled and non-topological structure. In this paper, we propose a novel hybrid architecture that fuses a TF–IDF-based autoencoder with a sentence-transformer encoder through a cross-attention mechanism. This combination leverages TF–IDF’s statistical sensitivity to repetitive attack signatures while enriching it with semantic understanding, allowing the latent representation to selectively incorporate both statistical anomalies and contextual semantic signals, thereby preserving the interpretability and clusterability of the learned space. We implement this architecture through a Wazuh-based SIEM deployment at the 5G network edge, demonstrating that unsupervised hybrid clustering can deliver effective CoAP anomaly detection in industrial IoT environments without labeled data, intrusive agents, or cloud-dependent processing addressing critical gaps in operational security for private 5G networks.