Run-Time Detection of Malwares via Dynamic Control-Flow Inspection

Abstract

Conventional approach of detecting malwares relies on static scanning of malware signature. However, it may not work on the malwares that use software protection methods such as encryption and packing with run-time decryption and unpacking. We propose a hardware-assisted malware detection system that detects malwares during program run time to complement the conventional approach. It searches for control flow-based signature of malware during program execution, therefore bypassing the protection method used by those malwares. A new hardware design is used to assist the collection of control flow information. We have implemented and evaluated a prototype system on top of a full-system simulator based on the Intel x86 architecture. The experimental results show that the system can successfully distinguish all 30 malware variants and other benign programs that we have randomly collected, and that the overall run-time performance overhead is negligible. In short, the study demonstrates that it is a viable approach to detect malware in run time using control flow-based signature.