Performance Maintenance Over Time of Random Forest-based Malware Detection Models

Abstract

It has been recognized that machine learning-based malware detection models, trained on features statically extractable from binary executable files, offer a number of promising benefits, such as the ability to detect malware that has not been previously encountered and an ability to re-train and adapt over time as threats evolve. Nevertheless, many academic studies of machine learning-based malware detection consider and evaluate performance on datasets that do not evolve with time, although it is recognized in practice that malware detection models will necessarily deteriorate in performance over time due to the emergence of novel malware threats. In this study, we make use of a large dataset comprised of the features extracted from malware/goodware executable samples in the very common Portable Executable (PE) format, that are orderable by time of first appearance, to analyze the deterioration of machine learning-based malware detection models over time from training. Of the large number of models we trained and then evaluated on later occurring subsets of the dataset, we note the relative strength of Random Forest to maintain predictive performance into the future. We then consider in greater depth, Random Forest-based models for malware detection, considering Random Forest hyperparameter choices to achieve better maintenance of performance and discuss the significance of the findings for PE malware detection approaches.