A novel stable feature selection algorithm for machine learning based intrusion detection system

Abstract

The advent of new technologies like artificial intelligence, and big data has influenced many cyber attackers to launch their attacks on the network. Hence researchers have already proposed Intrusion Detection Systems by incorporating machine learning as well. Building an effective IDS is still a challenging task because of low accuracy. Managing high dimensional data is another major problem that occurs in IDS. Hence in this paper, an efficient Machine Learning based Intrusion Detection System is developed by means of a novel stable feature selection strategy called IV-RFE. The proposed methodology aims to select only the relevant features that contribute to the attack, which is purely based on relative variance, and weight factor in combination with RFE. This methodology increases the performance in terms of accuracy and maintains a stable set of features. Previous studies only focussed on the feature selection strategy and their performance. The feature stability also has to be considered which is an equally important metric, especially in the field of Intrusion Detection Systems. Hence in the current study, an efficient ML based IDS is proposed which selects only the relevant and stable features. Experimental results also revealed that the proposed IV-RFE outperformed well for three attacks with respect to accuracy and stability metrics also. The results show that stability is also an important indicator in selecting the features in the field of Intrusion Detection Systems.