Sequence-Based Analysis of Static Probe Instrumentation Data for a VMM-Based Anomaly Detection System

Abstract

In this work, we propose a framework for a Virtual Machine Monitor (VMM)-based Anomaly Detection System (ADS). This framework uses a sequence-based analysis Hidden Markov Model (HMM) on static probe instrumentation data collected within the VMM. Long observations are split into multiple, uniformed-length, small sequences. The list of likelihood score of sequences in the new observation is compared to a reference list of likelihood scores created from a normal scenario dataset. Statistical distance values from both lists are used to predict the new observation anomaly status. We evaluated the effectiveness of the approach over multiple statistical distance measures and multiple sequence lengths. We also compared our sequence-based analysis results with a frequency-based analysis results that used the One-Class Support Vector Machine (OC-SVM). The results show that the HMM sequence-based analysis can distinguish normal datasets from anomalous datasets better than the OC-SVM frequency-based analysis.