Hide and Seek Game: A Machine Learning Approach for Detecting Malicious Samples in Analysis Environment

Abstract

In this work, we investigate whether malware understands the analysis environment. This analysis is carried out by executing a set of real malicious programs and benign samples on virtual and native machines. The result of execution is API sequence collected independently from virtual machines and host systems. In order to enhance the detection rate and accuracy, we have introduced four feature selection techniques. Thus, identified that feature reduction methods enhance the detection rate to a considerable extent. The experimental study depicted that while classifying malware and benign samples in virtual machines, most of the samples have misclassified, giving a clear indication that many malware samples remain dormant on identifying sandbox environment.