Evaluation of Run-Time Detection of Self-Replication in Binary Executable Malware

Abstract

This paper presents an overview and evaluation of a novel approach for proactive protection against both known and previously unknown self-replicating malicious software. Instead of deciphering and screening suspect code for signatures of known viruses the approach monitors the runtime behavior of binary compiled executable code by monitoring its system calls. The detection mechanism, which works from the perspective of the operating system, is based on identifying the unique self-replication behavior of executable malware via its system call sequences. Thus, the proposed approach provides a system that can detect self-replication attempts in malware without relying on the availability of a signature in a virus signature database and despite any level of encryption employed. An implementation of the proposed approach for Microsoft Windows operating system is described along with experimental results and a performance analysis