Statistical machine learning for network intrusion detection: a data quality perspective

In this paper, we present our research in applying statistical machine learning methods for network intrusion detection. With the advent of online distributed services, the issue of preventing network intrusion and other forms of information security failures is gaining prominence. In this work, we use two different algorithms for classification (decision trees and naive Bayes classifier) to build predictive models capable of distinguishing between 'bad' TCP/IP connections, called intrusions attacks, and 'good' normal TCP/IP connections. We investigate the effect of training the models using both clean and dirty data. The goal is to analyse the predictive power of network intrusion classification models trained with data of varying quality. The classifiers are contrasted with a clustering-based approach for comparison purposes.