A Hybrid Model for BGP Anomaly Detection Using Median Absolute Deviation and Machine Learning

Abstract

Detecting anomalies in the Border Gateway Protocol (BGP) has proved relevant in the cybersecurity field due to the protocol’s critical role in the Internet’s infrastructure. BGP vulnerabilities can lead to major disruptions and connectivity failures, highlighting the need for early detection to maintain stable and secure Internet services. To address this challenge, our article presents an enhanced version of our previously published Median Absolute Deviation (MAD) anomaly detection system. We introduce a novel dynamic threshold mechanism that significantly enhances anomaly detection performance in BGP, achieving superior accuracy and F1-score. Through a comparative analysis of machine learning (ML) classification models—including Random Forest, Extra Trees, XGBoost, LightGBM, and CatBoost—we demonstrate that integrating our MAD detection system with these ML models can improve anomaly detection significantly. Additionally, we explore how MAD performs when combined with neural networks such as RNN, GRU, and LSTM, providing a valuable comparison between machine learning and neural network-based approaches. We evaluate the models performance in well-known events, such as CodeRed 1 v2, Slammer, Nimda, the Moscow blackout, and the Telekom Malaysia (TMnet) misconfiguration. Our results show an overall accuracy of 0.99 and F1-score of 0.98, demonstrating the effective integration of MAD and ML models for the identification of security threats. Our approach enables proactive detection with minimal computational costs and reduced preprocessing, proving that efficient anomaly detection is achievable.