A Method of Constructing Malware Classification Dataset Using Clustering

Abstract

Machine learning, which automatically learns models from data, is receiving a lot of attention as a solution to cope with the increasing number of malicious codes every year. However, since most malicious codes are variants developed by recycling existing malicious codes, there is a problem that the model is easily overfitted to the training set compared to other domains. Previous studies have tried to remove the variants using labels provided by vaccines, but it can lead to indiscriminate removal of malicious codes since the vaccine label is inaccurate. Therefore, we propose a method of constructing a dataset by performing clustering and randomly selecting one from a cluster. To demonstrate that the proposed method of constructing training set can prevent overfitting and improve the generalization performance, we experimented with three training sets: a set that variants are not removed, a set that duplicated families are removed using labels, and a set that duplicated families are removed by the proposed method. To measure generalization performance, we experimented with six test sets constructed by the similarity to the training sets. It was confirmed that models learned from the training set constructed by the proposed method performed better on four test sets than the other models.