Web Content Outlier Mining using Machine Learning and Mathematical Approaches

Abstract

Due to the massive, dynamic and heterogeneous nature of the web, discovering outliers from the web is demanding than from the numeric dataset. On exploring for information in the web, the inappropriate irrelevant and redundant information may be retrieved to the user. So, it is a big challenge to get and access high quality information on the web effectively and efficiently without including irrelevant and redundant information. Mining web content outliers focus on mining inappropriate duplicate and irrelevant web pages from the other web pages under the same categories. Removing outliers from the web improves the accuracy of search results, decreases the complexity of time for indexing and complexity of time and saves the user time and effort. We applied the Latent Dirichlet Allocation method from the machine learning approaches and a mathematical approach named linear correlation method to move web content outliers. This system tends to improve F1-measure, accuracy results and reduce time complexity.