Similarity Measures and Text Documents Classfication Accuracies Using Benchmark Datasets

The total amount of text data information available on the web has been remarkably increasing the data accumulating and augmenting each day. Data and information that are available in high volume are not represented in a structured form that remains suitable for text processing. Text data mining is a subfield of data mining which aims at exploring the useful information from the recorded resources. Text data mining has the following key challenges namely high dimensionality, distance measures between data, achieving quality and classifier accuracies. The research work has attempted to addresses the key text data mining challenges by proposing high-dimensionality reduction novel techniques based on feature similarity functions. In the proposed design, the feature similarity measures are used to group features into clusters. From these feature clusters, an optimal transformation matrix is obtained using which the high dimension text corpus is projected to its equivalent low dimension. This low dimensionality text corpus can be used to implement text clustering and text classification efficiently.