Memory-efficient regular expression matching for Chinese network content audit

Abstract

When match against Chinese keyword for network content audit, one of the biggest problems is that there is interference of “noise characters”, it makes the traditional way using explicit string pattern to match infeasible. Regular expression matching can solve the problem perfectly, but the DFA-base approaches for regular expression matching will also encounter the problem of excessive memory usage. In this paper, we try to solve the problem encountered when applying regular expression to Chinese network content audit. We propose a regular expression rewriting techniques and grouping principle that can solve excessive memory usage problem in DFA-based approach. Our solution can make it possible to apply regular expression to Chinese network content audit.