Theoretic analysis of finite automata for memory-based pattern matching

In the midst of vastly numbered and quickly growing internet security threats, Network Intrusion Detection System (NIDS) [2] becomes more important to network security every day. Vital to effective NIDS is a multi-pattern matching engine which requires deterministic performance and adaptability to new threats [3]. Memory-based Deterministic Finite Automata (DFA) are ideal for pattern matching but have severe memory requirements [1] that make them difficult to implement. Many previous heuristic techniques have been proposed to reduce memory requirements, however in this paper, we aim to effectively understand the basic relationship between DFA characteristics and memory, in order to create minimal memory DFA implementations. We show what DFA characteristics either cause or reduce memory requirements, as well as how to optimize DFA to exploit those characteristics. Specifically, we introduce the concepts of State Independence and State Irregularity, which are DFA characteristics that can reduce memory waste and allow for memory reuse. Furthermore, we introduce DFA normalization which optimizes DFA to fully exploit these characteristics. Altogether this work serves as a source for how to extract and utilize DFA characteristics to create minimal memory implementations.