Protecting Run-Time Filters for Network Intrusion Detection Systems

Abstract

Network Intrusion Detection Systems (NIDS) examine millions of network packets searching for malicious traffic. Multi-gigabit line-speeds combined with growing databases of rules lead to dropped packets as the load exceeds the capacity of the device. Several areas of research have attempted to mitigate this problem through improving packet inspection efficiency, increasing resources, or reducing the examined population. A popular method for reducing the population examined is to employ run-time filters that can provide a quick check to determine that a given network packet cannot match a particular rule set. While this technique is an excellent method for reducing the population under examination, rogue elements can trivially bypass such filters with specially crafted packets and render the run-time filters effectively useless. Since the filtering comes at the cost of extra processing a filtering solution could actually perform worse than a non-filtered solution under such pandemic circumstances. To defend against such attacks, it is necessary to consider run-time filters as an independent anomaly detector capable of detecting attacks against itself. Such anomaly detection, together with judicious rate-limiting of traffic forwarded to full packet inspection, allows the detection, logging, and mitigation of attacks targeted at the filters while maintaining the overall improvements in NIDS performance garnered from using run-time filters.