A real time unsupervised NIDS for detecting unknown and encrypted network attacks in high speed network

Previously, Network Intrusion Detection Systems (NIDS) detected intrusions by comparing the behaviour of the network to the pre-defined rules or pre-observed network traffic, which was expensive in terms of both cost and time. Unsupervised machine learning techniques have overcome these issues and can detect unknown and complex attacks within normal or encrypted communication without any prior knowledge. NIDS monitors bytes, packets and network flow to detect intrusions. It is nearly impossible to monitor the payload of all packets in a high-speed network. On the other hand, the content of packets does not have sufficient information to detect a complex attack. Since the rate of attacks within encrypted communication is increasing and the content of encrypted packets is not accessible to NIDS, it has been suggested to monitor network flows. As most network intrusions spread within the network very quickly, in this paper we will propose a new real-time unsupervised NIDS for detecting new and complex attacks within normal and encrypted communications. To achieve having a real-time NIDS, the proposed model should capture live network traffic from different sensors and analyse specific metrics such as number of bytes, packets, network flows, and the time explicitly and implicitly, of packets and network flows, in the different resolutions. The NIDS will flag the time slot as an anomaly if any of those metrics passes the threshold, and it will send the time slot to the first engine. The first engine clusters different layers and dimensions of the network's behaviour and correlates the outliers to purge the intrusions from normal traffic. Detecting network attacks, which produce a huge amount of network traffic (e.g. DOS, DDOS, scanning) was the aim of proposing the first engine. Analysing statistics of network flows increases the feasibility of detecting intrusions within encrypted communications. The aim of proposing the second engine is to conduct a deeper analysis and correlate the traffic and behaviour of Bots (current attackers) during DDOS attacks to find the Bot-Master.