A graph-based model for malicious code detection exploiting dependencies of system-call groups

Abstract

In this paper, we propose a graph-based algorithmic technique for malware detection. More precisely, we utilize the system-call dependency graphs (or, for short, ScD graphs), obtained by capturing taint analysis traces and a set of various similarity metrics in order to detect whether an unknown test sample is a malicious or a benign one. For the sake of generalization, we decide to empower our model against strong mutations by applying our detection technique on a weighted directed graph resulting from ScD graph after grouping disjoint subsets of its vertices. Additionally, we propose the Δ-Similarity metric, which is based on the Euclidean distance operating on the in-degree and out-degree of ScD's nodes along with their corresponding weights, distinguishing thus graph-representations of malware and benign software. Finally, we evaluate the potentials of our detection model and show that its performance makes it competing to other detection models.