Concept Drift Adaptation in Intrusion Detection Systems Using Ensemble Learning

Abstract

The primary challenge of intrusion detection systems (IDS) is to rapidly identify new attacks, learn from the adversary, and update the intrusion detection immediately. IDS operate in dynamic environments subjected to evolving data streams where data may come from different distributions. This is known as the problem of concept drift. Today's IDS though are equipped with deep learning algorithms most of the times fail to identify concept drift. This paper presents a technique to detect and adapt to concept drifts in streaming data with a large number of features often seen in IDS. The study modifies extreme gradient boosting (XGB) algorithm for adaptability of drifts and optimization for large datasets in IDS. The primary objective is to reduce the number of ‘false positives' and ‘false negatives' in the predictions. The method is tested on streaming data of smaller and larger sizes and compared against non-adaptive XGBoost and logistic regression.