Automated text document categorization

Abstract

During the last two decades the number of text documents in digital form has grown enormously. It is necessary to categorize documents into topics and sub topics for easy retrieval. Manual categorization of text documents can be done only by experts and it is a time consuming task. As a consequence, it is of great practical importance to be able to automatically organize and classify documents. There are two approaches, rule-based and machine learning-based, that are used to automate classification task. Both have some limitations. Rules may conflict each other and have to be reconstructed when a target domain changes, are such two limitations in the rule based approaches. Machine learning approaches require proper training data and they do not accountable with the classification results. Motivated by such limitations, this paper proposes a Latent Dirichlet Allocation (LDA) based approach to automatically classify text documents. In order to develop and test the proposed approach on a realistic set up, ACM (Association for Computing Machinery) Computing Classification System (CCS) is selected as the target platform and 9100 computer science related articles categorized under ACM-CCS were selected. The experimental results show that the proposed approach is effective for classifying text documents and is applicable to a domain with large number of categories in multiple levels.