A comparative study of machine learning and deep learning models in binary and multiclass classification for intrusion detection systems

Abstract

Network infrastructure evolution has significantly expanded the attack surface, leading to increasingly complex and sophisticated cybersecurity threats. Traditional rule-based intrusion detection systems (IDS) often fail to detect emerging attack vectors, prompting the need for intelligent, data-driven approaches. This study evaluates and compares the performance of machine learning (ML) and deep learning (DL) models for network intrusion detection. Two publicly available datasets were utilized: a binary-labeled software-defined networking (SDN) dataset and a multiclass industrial control system dataset based on the IEC 60870-5-104 protocol. Preprocessing steps included normalization, label encoding, and a 70:10:20 train-validation-test split. Seven models, Random Forest, Decision Tree, K-Nearest Neighbors, XGBoost, Convolutional Neural Network, Gated Recurrent Unit, and Long Short-Term Memory, were trained and evaluated using precision, recall, and F1-score. The Random Forest model achieved the highest F1-score of 93.57 % on the IEC 60870-5-104 dataset, while XGBoost attained a near-perfect F1-score of 99.97 % on the SDN dataset. These results outperform comparable models in the literature and offer practical insights for selecting effective IDS solutions based on classification type and dataset structure.