CHEAT: A Large-Scale Dataset for Detecting CHatGPT-writtEn AbsTracts

IF 7.5 3区计算机科学 Q1 COMPUTER SCIENCE, INFORMATION SYSTEMS

IEEE Transactions on Big Data Pub Date : 2025-01-30 DOI:10.1109/TBDATA.2025.3536929

Peipeng Yu;Jiahan Chen;Xuan Feng;Zhihua Xia

{"title":"CHEAT: A Large-Scale Dataset for Detecting CHatGPT-writtEn AbsTracts","authors":"Peipeng Yu;Jiahan Chen;Xuan Feng;Zhihua Xia","doi":"10.1109/TBDATA.2025.3536929","DOIUrl":null,"url":null,"abstract":"The powerful ability of ChatGPT has caused widespread concern in the academic community. Malicious users could synthesize dummy academic content through ChatGPT, which is extremely harmful to academic rigor and originality. The need to develop ChatGPT-written content detection algorithms calls for large-scale datasets. In this paper, we initially investigate the possible negative impact of ChatGPT on academia, and present a large-scale CHatGPT-writtEn AbsTract dataset (CHEAT) to support the development of detection algorithms. In particular, the ChatGPT-written abstract dataset contains 35,304 synthetic abstracts, with <inline-formula><tex-math>$Generation$</tex-math></inline-formula>, <inline-formula><tex-math>$Polish$</tex-math></inline-formula>, and <inline-formula><tex-math>$Fusion$</tex-math></inline-formula> as prominent representatives. Based on these data, we perform a thorough analysis of the existing text synthesis detection algorithms. We show that ChatGPT-written abstracts are detectable with well-trained detectors, while the detection difficulty increases with more human guidance involved.","PeriodicalId":13106,"journal":{"name":"IEEE Transactions on Big Data","volume":"11 3","pages":"898-906"},"PeriodicalIF":7.5000,"publicationDate":"2025-01-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Big Data","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10858415/","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}

引用次数: 0

Abstract

The powerful ability of ChatGPT has caused widespread concern in the academic community. Malicious users could synthesize dummy academic content through ChatGPT, which is extremely harmful to academic rigor and originality. The need to develop ChatGPT-written content detection algorithms calls for large-scale datasets. In this paper, we initially investigate the possible negative impact of ChatGPT on academia, and present a large-scale CHatGPT-writtEn AbsTract dataset (CHEAT) to support the development of detection algorithms. In particular, the ChatGPT-written abstract dataset contains 35,304 synthetic abstracts, with

$Generation$

$Polish$

, and

$Fusion$

as prominent representatives. Based on these data, we perform a thorough analysis of the existing text synthesis detection algorithms. We show that ChatGPT-written abstracts are detectable with well-trained detectors, while the detection difficulty increases with more human guidance involved.

查看原文本刊更多论文

CHEAT：用于检测CHatGPT-writtEn摘要的大规模数据集

ChatGPT的强大能力引起了学术界的广泛关注。恶意用户可以通过ChatGPT合成虚假的学术内容，这对学术严谨性和原创性是极其有害的。开发chatgpt编写的内容检测算法需要大规模的数据集。在本文中，我们初步研究了ChatGPT对学术界可能产生的负面影响，并提出了一个大规模的ChatGPT - written AbsTract dataset （CHEAT）来支持检测算法的开发。特别是，chatgpt编写的摘要数据集包含35304个合成摘要，其中$Generation$, $Polish$和$Fusion$是突出的代表。基于这些数据，我们对现有的文本合成检测算法进行了深入的分析。我们表明，chatgpt编写的摘要可以被训练有素的检测器检测到，而检测难度随着人工指导的增加而增加。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

IEEE Transactions on Big Data Multiple-

CiteScore

11.80

自引率

2.80%

发文量

114

期刊介绍： The IEEE Transactions on Big Data publishes peer-reviewed articles focusing on big data. These articles present innovative research ideas and application results across disciplines, including novel theories, algorithms, and applications. Research areas cover a wide range, such as big data analytics, visualization, curation, management, semantics, infrastructure, standards, performance analysis, intelligence extraction, scientific discovery, security, privacy, and legal issues specific to big data. The journal also prioritizes applications of big data in fields generating massive datasets.