Semi-supervised traceability analysis of investigative scanners of darknet traffic

Abstract

Darknet, an unused IP address space on the Internet, has led to significant research advances in the analyses of global scanning activities, predictions of incoming cyber threats, and the classification of scanning patterns in unsolicited network traffic. However, most darknet traffic research has focused on classification methods that rely on supervised learning, or on unsupervised methods that require further expert effort. To study the applicability of semi-supervision for darknet traffic analysis, we propose a semi-supervised framework that efficiently clusters and classifies scanner behaviors based on existing knowledge for the traceability analysis of investigative scanners on the darknet. The framework utilizes a word embedding model to represent similarly behaving scanners in close proximity in the vector space, followed by a semi-supervised clustering step that incorporates partial labels of known scanners. We validate the framework by combining two publicly available darknet traffic datasets: CAIDA, providing labeled data for semi-supervision, and NICT, that offers a larger set of unlabeled data for analysis. Experimental results demonstrated that integrating semi-supervised learning into darknet traffic analysis improves the interpretability of diverse scanning behaviors and enhances scalability, offering a three-fold speedup in overall runtime compared to the existing sliding window approach. By reducing reliance on fully labeled datasets, the framework facilitates large-scale threat intelligence while allowing for the smooth integration of ever-growing domain knowledge pertaining to darknet traffic. Future research can further refine the model by incorporating additional classes of darknet scanners and expanding the applicability of the model to real-time darknet traffic analysis.