Artificially Generated Text Fragments Search in Academic Documents

Pub Date : 2024-03-11 DOI:10.1134/S1064562423701211

G. M. Gritsay, A. V. Grabovoy, A. S. Kildyakov, Yu. V. Chekhovich

{"title":"Artificially Generated Text Fragments Search in Academic Documents","authors":"G. M. Gritsay, A. V. Grabovoy, A. S. Kildyakov, Yu. V. Chekhovich","doi":"10.1134/S1064562423701211","DOIUrl":null,"url":null,"abstract":"<p>Recent advances in text generative models make it possible to create artificial texts that look like human-written texts. A large number of methods for detecting texts obtained using large language models have already been developed. However, improvement of detection methods occurs simultaneously with the improvement of generation methods. Therefore, it is necessary to explore new generative models and modernize existing approaches to their detection. In this paper, we present a large analysis of existing detection methods, as well as a study of lexical, syntactic, and stylistic features of the generated fragments. Taking into account the developments, we have tested the most qualitative, in our opinion, methods of detecting machine-generated documents for their further application in the scientific domain. Experiments were conducted for Russian and English languages on the collected datasets. The developed methods improved the detection quality to a value of 0.968 on the F1-score metric for Russian and 0.825 for English, respectively. The described techniques can be applied to detect generated fragments in scientific, research, and graduate papers.</p>","PeriodicalId":0,"journal":{"name":"","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-03-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"","FirstCategoryId":"100","ListUrlMain":"https://link.springer.com/article/10.1134/S1064562423701211","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 0

Abstract

Recent advances in text generative models make it possible to create artificial texts that look like human-written texts. A large number of methods for detecting texts obtained using large language models have already been developed. However, improvement of detection methods occurs simultaneously with the improvement of generation methods. Therefore, it is necessary to explore new generative models and modernize existing approaches to their detection. In this paper, we present a large analysis of existing detection methods, as well as a study of lexical, syntactic, and stylistic features of the generated fragments. Taking into account the developments, we have tested the most qualitative, in our opinion, methods of detecting machine-generated documents for their further application in the scientific domain. Experiments were conducted for Russian and English languages on the collected datasets. The developed methods improved the detection quality to a value of 0.968 on the F1-score metric for Russian and 0.825 for English, respectively. The described techniques can be applied to detect generated fragments in scientific, research, and graduate papers.

Abstract Image

查看原文

在学术文献中搜索人工生成的文本片段

摘要文本生成模型方面的最新进展使创建与人类书写文本相似的人工文本成为可能。目前已开发出大量使用大型语言模型检测文本的方法。然而，检测方法的改进与生成方法的改进是同步进行的。因此，有必要探索新的生成模型，并更新现有的检测方法。在本文中，我们对现有的检测方法进行了大量分析，并对生成片段的词法、句法和文体特征进行了研究。考虑到发展情况，我们测试了我们认为最有质量的机器生成文档检测方法，以便在科学领域进一步应用。我们在收集到的数据集上对俄语和英语进行了实验。所开发的方法提高了检测质量，俄语的 F1 分数指标值为 0.968，英语的 F1 分数指标值为 0.825。所述技术可用于检测科学、研究和研究生论文中生成的片段。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文