Realism versus Performance for Adversarial Examples Against DL-based NIDS

The application of deep learning-based (DL) network intrusion detection systems (NIDS) enables effective automated detection of cyberattacks. Such models can extract valuable features from high-dimensional and heterogeneous network traffic with minimal feature engineering and provide high accuracy detection rates. However, it has been shown that DL can be vulnerable to adversarial examples (AEs), which mislead classification decisions at inference time, and several works have shown that AEs are indeed a threat against DL-based NIDS. In this work, we argue that these threats are not necessarily realistic. Indeed, some general techniques used to generate AE manipulate features in a way that would be inconsistent with actual network traffic. In this paper, we first implement the main AE attacks selected from the literature (FGSM, BIM, PGD, NewtonFool, CW, DeepFool, EN, Boundary, HSJ, ZOO) for two different datasets (WSN-DS and BoT-IoT) and we compare their relative performance. We then analyze the perturbation generated by these attacks and use the metrics to establish a notion of "attack unrealism". We conclude that, for these datasets, some of these attacks are performant but not realistic.