Intelligent grouping algorithms for regular expressions in deep inspection

Abstract

Deep inspection is widely used to identify network traffic. Due to the complexity of payload in traffic, regular expressions are becoming the preferred choice to specify the rules of deep inspection. Compiling a set of regular expressions into one Deterministic Finite Automata (DFA) often leads to state explosion, which means huge or even impractical memory cost. Distributing a set of regular expressions into multiple groups and building DFAs independently for each group mitigates the problem, but the previous grouping algorithms are either brute-force or locally optimal and thus not efficient in practice. Inspired by the Intelligent Optimization Algorithms, we propose new grouping algorithms based on Genetic Algorithm and Ant Colony Optimization algorithm, to effectively solve the problem of state explosion by acquiring the global optimum tradeoff between memory consumption and the number of groups. Besides, to accelerate the execution speed of the intelligent grouping algorithms, we employ and improve an approximation algorithm that estimates the DFA states according to the conflicting relationship between each pair of regular expressions. Experimental results verify that our solutions save around 25% memory consumption or reduce around 20% of group number compared with existing popular grouping algorithms.