Zero day malware detection with Alpha: Fast DBI with Transformer models for real world application

Abstract

The effectiveness of an AI model in accurately classifying novel malware hinges on the quality of the features it is trained on, which in turn depends on the effectiveness of the analysis tool used. Peekaboo, a Dynamic Binary Instrumentation (DBI) tool, defeats malware evasion techniques to capture authentic behavior at the Assembly (ASM) instruction level. This behavior exhibits patterns consistent with Zipf’s law, a distribution commonly seen in natural languages, making Transformer models particularly effective for binary classification tasks.

We introduce Alpha, a framework for zero-day malware detection that leverages Transformer models, Support Vector Machines (SVMs) and ASM language features. Alpha is trained on malware and benign software data extracted at the ASM level, enabling it to detect entirely new malware samples with exceptional accuracy. Alpha eliminates any common functions from the test samples that are in the training dataset. This forces the model to rely on contextual patterns and novel ASM instruction combinations to detect malicious behavior, rather than memorizing familiar features. By combining the strengths of DBI, ASM analysis, and Transformer architectures, Alpha offers a powerful approach to proactively addressing the evolving threat of malware. Alpha demonstrates excellent accuracy for Ransomware, Worms and APTs with flawless classification for both malicious and benign samples. The results highlight the model’s exceptional performance in detecting truly new malware samples.