unFlowS: An Unsupervised Construction Scheme of Flow Spectrum for Network Traffic Detection

Abstract

In recent years, the construction of behavior-based analysis models is hindered by issues such as insufficient data, difficulty in labeling, and the complexity of behavior types. In reality, specific cyber threats often require manual analysis of raw network traffic, which is a complex and inefficient process. Flow spectrum can simplify the complex analysis process of raw network flow by mapping it from a high-dimensional space to a one-dimensional spectral space. However, the existing flow spectrum cannot adapt to the open-world scenarios and behavior-based detection for unknown cyber threats. To address these challenges, we propose a new flow spectrum construction scheme, named unFlowS, to effectively represent network flows and assist analysts to understand the behaviors of network traffic. unFlowS-Net, an unsupervised flow-based detection model we designed as the core of our scheme, can transform network flows into spectral lines. It makes unFlowS possible to detect unknown cyber threats. We further build spectral vectors for spectral lines generated by network flow sets, enabling the visualization of network behaviors within a period of time and automatic behavior-based detection. Experimental results demonstrated that unFlowS-Net can achieve better performance than state-of-the-art methods on unsupervised flow-based detection. Based on spectral vectors, not only can it intuitively display the network behavior characteristic of the target host, but also automatically detect suspicious network behaviors.