Approximate search engine optimization for directory service

Abstract

Today, in many practical e-commerce systems, the real stored data usually are short strings, such as names, addresses, or other information. Searching data within these short strings is not the same as searching within longer strings. General search engines try their best to scan all long strings (or articles) quickly, and find out the places that match the search conditions. Some great online search algorithms (such as "agrep" as used inside glimpse, or "cgrep " as used inside compressed indices, or 'NR-grep') are proposed for searching without any indices in the sub-linear time O(n). However, for short strings (n is small), the practical performance of algorithms of O(n) and O(n) are much the same. Therefore, suitable indices are necessary to optimize the performance of the search engine. On the other hand, directory services are more and more important because of its optimization for searching data. The data stored in directory servers are almost short strings. The approximate search engine for directory service must take the properties of short strings into considerations. In our previous research, we have designed one approximate search engine especially for short strings by using filters to filter out the possible short strings, and then checking for the answers. However the performance of the previous search engine needs to be enhanced. In this paper, we propose new architecture and algorithm to optimize the performance of searching for directory service.