Online and incremental mining of separately-grouped Web access logs

Abstract

The rising popularity of electronic commerce makes data mining an indispensable technology for business competitiveness. The World Wide Web provides abundant raw data in the form of Web access logs, Web transaction logs and Web user profiles. Without data mining tools, it is impossible to make any sense of such massive data. We focus on Web usage mining because it deals most appropriately with understanding user behavioral patterns which is the key to successful customer relationship management. Previous work dealt separately with specific issues of Web usage mining and made assumptions without taking a holistic view and thus, had limited practical applicability. We formulate a novel and more holistic version of Web usage mining termed transactionized logfile mining (TRALOM) to effectively and correctly identify transactions as well as to mine useful knowledge from Web access logs. We also introduce a new data structure, called the WebTrie, to efficiently hold useful preprocessed data so that TRALOM can be done in an online and incremental fashion. Experiments conducted on real Web server logs verify the usefulness and practicality of our proposed techniques.