Feature Rankers to Predict Classification Performance of Unsupervised Intrusion Detectors

Abstract

An anomaly-based Intrusion Detection System (IDS) consists of a monitor and a binary classifier, in which monitored system indicators are fed into a Machine Learning (ML) algorithm that detects anomalies due to attacks. Building such an IDS for a target system requires first to define a strategy to monitor features, then to select and evaluate many ML algorithms to find the most suitable candidate. Noticeably, features that do not fluctuate enough when attacks happen will negatively affect detection performance. In this paper we propose a strategy to predict the classification performance of unsupervised anomaly-based intrusion detectors without any knowledge or execution of the ML algorithm. We experimentally verify that individual scores assigned to features by filter and wrapper-based feature rankers can be used to predict the classification performance of anomaly detectors. Particularly, we detail, explain and motivate how feeding scores of feature rankers into a Random Forest regressor allows predicting the value of common evaluation metrics for anomaly detectors as F1 or MCC with average of relative residuals lower than 15%, and how to take advantage of our prediction strategy in different scenarios.