Individual Packet Features are a Risk to Model Generalization in ML-Based Intrusion Detection

Abstract

Machine learning is increasingly employed for intrusion detection in IoT networks. This letter provides the first empirical evidence of the risks associated with modeling network traffic using individual packet features (IPF). Through a comprehensive literature review and novel experimental case studies, we identify critical limitations of IPF, such as information leakage and low data complexity. We offer the first in-depth critique of IPF-based detection systems, highlighting their risks for real-world deployment. Our results demonstrate that IPF-based models can achieve deceptively high detection rates (up to 100% in some cases), but these rates fail to generalize to new datasets, with performance dropping by more than 90% in cross-session tests. These findings underscore the importance of considering packet interactions and contextual information, rather than relying solely on IPF, for developing robust and reliable intrusion detection systems in IoT environments.