Design of a Machine Learning Based Intrusion Detection Framework and Methodology for IoT Networks

Abstract

Traditional security solutions may not be always possible in IoT systems because of the resource constraint in IoT devices. Intrusion detection in IoT systems using Machine Learning (ML) techniques can be an effective measure in combating attacks. While most researchers focus on small datasets for ease of processing and training, model generalizability and accuracy can be improved significantly by training and fine-tuning models with big datasets. In this paper we proposed, implemented and evaluated a software framework using Hadoop cluster to store big dataset and PySpark library to train anomaly detection and attack classification models for securing IoT networks. We used the bigger version of the UNSW BoT IoT public dataset to fine-tune the ML-based models. With feature engineering and hyper-parameter tuning of anomaly detection model parameters, an accuracy of 96.3 % was achieved with maximum accuracy of 99. 9% in Reconnaissance attack detection.