Design consideration of Network Intrusion detection system using Hadoop and GPGPU

Abstract

Modern computing has primarily shifted towards the distributed environment using commodity resources which results in increase in data and its security concern. This paper deals with design consideration of Network Intrusion Detection System (NIDS) based on the Hadoop framework and acceleration of its performance by using General Purpose Graphical Processing Unit (GPGPU). The large volume of data from an entire infrastructure is assigned to Hadoop framework and intrusion detections are carried out on GPGPU. This approach improves NIDS performance and it enables to provide quick response to various attacks on the network. In order to perform the general purposed computation on the GPU, NVidia provides the Compute Unified Device Architecture (CUDA) which is a parallel programming model which performs high-end complex operations using GPU. In order to process large volumes of data in distributed networks, Hadoop framework has to configure with various supporting ecosystems like Flume, Pig, Hive and HBase. These ecosystems enable the Hadoop framework to handle streaming data on the network and large log files on servers. The proposed system is capable of performing analytics over intrusion pattern and their behavior on the network, which helps a network administrator to configure network security policy and settings. Analytics over intrusion is done by using a Score-Weight approach called as Pattern Frequency Inverse Cluster Frequency (PF-ICF). The design consideration of accelerated NIDS is a solution towards the performance issues of various NIDS that faces due to the large volumes of the network traffic.