Investigation of network intrusion detection using data visualization methods

Abstract

There are numerous sources of network intrusion detection data, for example, network traffic, system host logs, user activity, such as mail or browsing, use of smart devices and similar. All these data come in big volume, velocity and variety. Analysis of such data is essential for making anomaly detection and intrusion prevention decisions. Common data processing steps, following the acquisition of data and pre-processing, are data reduction and projection. These steps help to reduce the number of dimensions, and visualization, which enables observation of distinct features in real time. Projection and visualisation, further discussed in this paper are required for better understanding of contained intrusion phenomena, such as data theft, malware activity or hacking attempts. Machine learning enables reduction of data complexity, supports discovery of anomalies and speedups related decision-making. Visualization helps further understand data by elaborating the well-hidden data properties and features. Numerous methods of multidimensional data visualization are currently available to assist data scientist or information security analyst in the broad landscape of intrusion data analysis. For simplicity, visualization methods in this paper are categorized as direct, linear projection, non-linear projection and other. Attention is drawn to linear projection, in particular principal components analysis, helping to select the most informative dimensions of the data. Principal Component analysis provide indication of anomalies of network traffic. Decision Tree method is utilized to provide decision criteria for anomaly recognition as an intrusion. Investigation in this research demonstrates that combination of PCA and Decision Tree methods allows classification of intrusions such as Smurf, Satan, Neptune, Portsweep, Ppsweep with probabilities higher than 95% with depth of tree set to 4 and number of PCA components set to 10. Nevertheless, Nmap and Teardrop intrusions are classified purely, therefore deeper Decision Tree is needed to increase classification accuracy.