Decluttering the Internet with Article Summarization, Classification, and Recommendation using Natural Language Processing

Abstract

The explosion of information available on the internet has led to an overwhelming amount of data that is available to users. The ability to efficiently consume and comprehend this information is crucial for productivity and knowledge expansion. However, only a few of researches focus on providing e-readers not only with comprehensive classification but also relevant recommendations based on the reader's history. The paper has been implemented in pursuit of facilitating the consumption of information with increased coherency and efficiency, leading to a proliferation in productivity in the fields of both professional and informal research using an easy-to-use web-based dashboard, the project works in 3 broad tenets. BERT Algorithm (Bidirectional Encoder Representations from Transformers) for information summarization, followed by Latent Dirichlet allocation (LDA) Algorithm for text classification and Collaborative filtering for recommending further articles for the user's knowledge expansion.