SemAC algorithm based on the tag semantic distance in network service information extraction

Abstract

The paper analyzed the methods of extracting information from the network traffic, among which the information extraction based on the Tag marker was studied in depth. Tags, which contain specific core words, can be considered as markers of different types of information. Due to the non-standardized nature the Tag marker, a Tag often has lots of interference characters and the traditional pattern matching algorithm may leads to mismatch or poor performance. After analyzing the network service tag characteristics, we presented the Tag word segmenting method and the concept of Tag semantic distance. By introducing the tag semantic distance concept, we improved the Aho-Corasick algorithm to identify the tags which we concerned in extracting network service information. This improved algorithm, SemAC, is based on tag words segmenting and semantic distance calculation by dual-way scanning on words-map. Using this improved AC algorithm based on pre-defined core words, we successfully identified the tags and extracted what we need from the packets stream of web sites. At the end of this paper time complexity is analyzed.