Reinforcing network security by converting massive data flow to continuous connections for IDS

Processing massive data flow in intrusion detection systems (IDS) become a serious challenge. It is considered as a major deficiency while handling heterogeneous and non-stationary data stream to uncover anomaly in the online operational mode. This paper proposes a novel online method that constructs connections from the massive data flow for evaluating IDS models. The proposed method overcomes this challenge by using a queuing concept of dynamic window size. It captures network traffic and hosts events constantly and handles them synchronously within time slot windows inside the queue in order to construct connection vectors based on certain features. We have evaluated the method in offline mode using DARPA dump data flow and in online mode using a simulated network at the university campus. In addition, we have evaluated our IDS model using the constructed connections to proof the feasibility and plausibility of the proposed method in IDS area. The performance evaluation confirms that, the proposed method is able to operate in offline as well online modes efficiently. Moreover, constructed connections are very adequate for training and evaluating IDS models.