自然语言处理应用中预处理算法的有效性

2020 IEEE International Conference on Problems of Infocommunications. Science and Technology (PIC S&T) Pub Date : 2020-10-06 DOI:10.1109/PICST51311.2020.9467919

Kirill Smelyakov, Danil Karachevtsev, Denis Kulemza, Yehor Samoilenko, Oleh Patlan, A. Chupryna

{"title":"自然语言处理应用中预处理算法的有效性","authors":"Kirill Smelyakov, Danil Karachevtsev, Denis Kulemza, Yehor Samoilenko, Oleh Patlan, A. Chupryna","doi":"10.1109/PICST51311.2020.9467919","DOIUrl":null,"url":null,"abstract":"this article is devoted to identifying features and analyzing the effectiveness of stop word removal, stemming and text lemmatization algorithms that are used in the widely used NLTK and Spacy libraries when preparing data in Natural Language Processing applications in determining the publication topic; work is based on the analysis of the results of numerous experiments on the application of the considered text processing algorithms for such common topics of web publications as sports, economics and news.","PeriodicalId":123008,"journal":{"name":"2020 IEEE International Conference on Problems of Infocommunications. Science and Technology (PIC S&T)","volume":"44 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2020-10-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"4","resultStr":"{\"title\":\"Effectiveness of Preprocessing Algorithms for Natural Language Processing Applications\",\"authors\":\"Kirill Smelyakov, Danil Karachevtsev, Denis Kulemza, Yehor Samoilenko, Oleh Patlan, A. Chupryna\",\"doi\":\"10.1109/PICST51311.2020.9467919\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"this article is devoted to identifying features and analyzing the effectiveness of stop word removal, stemming and text lemmatization algorithms that are used in the widely used NLTK and Spacy libraries when preparing data in Natural Language Processing applications in determining the publication topic; work is based on the analysis of the results of numerous experiments on the application of the considered text processing algorithms for such common topics of web publications as sports, economics and news.\",\"PeriodicalId\":123008,\"journal\":{\"name\":\"2020 IEEE International Conference on Problems of Infocommunications. Science and Technology (PIC S&T)\",\"volume\":\"44 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2020-10-06\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"4\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2020 IEEE International Conference on Problems of Infocommunications. Science and Technology (PIC S&T)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/PICST51311.2020.9467919\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2020 IEEE International Conference on Problems of Infocommunications. Science and Technology (PIC S&T)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/PICST51311.2020.9467919","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 4

摘要

在自然语言处理应用中，当准备数据时，广泛使用的NLTK和space库中使用的停止词去除、词干提取和文本排列算法用于确定出版主题，本文致力于识别这些算法的特征并分析其有效性;工作是基于对许多实验结果的分析，这些实验是关于将所考虑的文本处理算法应用于体育、经济和新闻等网络出版物的常见主题的。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Effectiveness of Preprocessing Algorithms for Natural Language Processing Applications

this article is devoted to identifying features and analyzing the effectiveness of stop word removal, stemming and text lemmatization algorithms that are used in the widely used NLTK and Spacy libraries when preparing data in Natural Language Processing applications in determining the publication topic; work is based on the analysis of the results of numerous experiments on the application of the considered text processing algorithms for such common topics of web publications as sports, economics and news.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

2020 IEEE International Conference on Problems of Infocommunications. Science and Technology (PIC S&T)

自引率

0.00%

发文量