Effectiveness of Preprocessing Algorithms for Natural Language Processing Applications

Abstract

this article is devoted to identifying features and analyzing the effectiveness of stop word removal, stemming and text lemmatization algorithms that are used in the widely used NLTK and Spacy libraries when preparing data in Natural Language Processing applications in determining the publication topic; work is based on the analysis of the results of numerous experiments on the application of the considered text processing algorithms for such common topics of web publications as sports, economics and news.