A novel malware variants detection method based On function-call graph

Code obfuscation plays a significant role in metamorphic malware. Moreover, identifying a metamorphic malware variant is a challenge task, because its obfuscation engine can easily generate various variants with different forms while maintaining the same functionality to escape detection. This paper presents a novel approach to recognize metamorphic malware based on programs' function-call graphs. Graph-coloring and cosine similarity techniques are used to measure the similarity of two programs on the basis of function-call graph. Experimental results have shown that the proposed method can accurately detect the metamorphic malware variants.