On the right choice of data from popular datasets for Internet traffic classification

Abstract

Machine learning (ML) models used to analyze Internet traffic, similar to models in all other fields of ML, need to be fed by training datasets. Many such sets consist of labeled samples of the collected traffic data from harmful and benign traffic classes captured from the actual traffic. Since the traffic recording tools capture all the transmitted data, they contain much information related to the registration process that is irrelevant to the actual traffic class. Moreover, they are not fully anonymized. Thus, there is a need to preprocess the data before proper modeling, which should always be addressed in related studies, but often, this is not done. In our paper, we focus on the dependence of the efficiency of threat detection ML models by selecting the appropriate data samples from the training sets during preprocessing. We are analyzing three popular datasets: USTC-TFC2016, VPN-nonVPN, and TOR-nonTOR, which are widely used in traffic classification, security, and privacy-enhancing technologies research. We show that some choices of data sample pieces, although maximizing the model’s efficiency, would not result in similar outcomes in the case of traffic data other than the learning set. The reason is that, in these cases, models are biased due to learning incidental correlations that appear in the datasets used for training the model, introduced by auxiliary data related to the network traffic capturing and transmission process. They are present in popular datasets but may never appear in traffic data. Consequently, the models trained on such datasets, without any preprocessing and anonymization, would never reach the accuracy levels of the training data. Our paper introduces five consecutive levels of anonymization of the traffic data and points out that only the highest provide correct learning results. We validate the results by applying decision trees, random forests, and extra tree models. Having found the optimal part of the header data that may safely be used, we focus on the length of the remaining part of the traffic data to find its minimal length, which preserves good detection accuracy.