比较人工智能探测器:评估性能和效率

Jeremie Busio Legaspi, Roan Joyce Ohoy Licuben, Emmanuel Alegado Legaspi, Joven Aguinaldo Tolentino
{"title":"比较人工智能探测器:评估性能和效率","authors":"Jeremie Busio Legaspi, Roan Joyce Ohoy Licuben, Emmanuel Alegado Legaspi, Joven Aguinaldo Tolentino","doi":"10.30574/ijsra.2024.12.2.1276","DOIUrl":null,"url":null,"abstract":"The widespread utilization of AI tools such as ChatGPT has become increasingly prevalent among learners, posing a threat to academic integrity. This study seeks to evaluate capability and efficiency of AI detection tools in distinguishing between human-authored and AI-generated works. Three-paragraph works on “AutoCAD and Architecture” were generated through ChatGPT, and three human-written works were subjected to evaluation. AI detection tools such as GPTZero, Copyleaks and Writer AI were used to evaluate these paragraphs. Parameters such as “Human/Human Text/Human Generated Text” and “AI/AI Content Detected” were used to evaluate the performance of the three AI detection tools in evaluating outputs. Findings indicate that GPT Zero and Copyleaks have higher reliability in determining human-authored work and AI generated work while Writer AI showed a notable content classification of “Human Generated Content” on all tested outputs showing less sensitivity on determining human-authored work and AI generated work. Findings indicate that the use of Artificial Intelligence as an AI detection tool should be accompanied with thorough validation and cross-referencing of results.","PeriodicalId":14366,"journal":{"name":"International Journal of Science and Research Archive","volume":"1 12","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-07-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Comparing ai detectors: evaluating performance and efficiency\",\"authors\":\"Jeremie Busio Legaspi, Roan Joyce Ohoy Licuben, Emmanuel Alegado Legaspi, Joven Aguinaldo Tolentino\",\"doi\":\"10.30574/ijsra.2024.12.2.1276\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"The widespread utilization of AI tools such as ChatGPT has become increasingly prevalent among learners, posing a threat to academic integrity. This study seeks to evaluate capability and efficiency of AI detection tools in distinguishing between human-authored and AI-generated works. Three-paragraph works on “AutoCAD and Architecture” were generated through ChatGPT, and three human-written works were subjected to evaluation. AI detection tools such as GPTZero, Copyleaks and Writer AI were used to evaluate these paragraphs. Parameters such as “Human/Human Text/Human Generated Text” and “AI/AI Content Detected” were used to evaluate the performance of the three AI detection tools in evaluating outputs. Findings indicate that GPT Zero and Copyleaks have higher reliability in determining human-authored work and AI generated work while Writer AI showed a notable content classification of “Human Generated Content” on all tested outputs showing less sensitivity on determining human-authored work and AI generated work. Findings indicate that the use of Artificial Intelligence as an AI detection tool should be accompanied with thorough validation and cross-referencing of results.\",\"PeriodicalId\":14366,\"journal\":{\"name\":\"International Journal of Science and Research Archive\",\"volume\":\"1 12\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-07-30\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"International Journal of Science and Research Archive\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.30574/ijsra.2024.12.2.1276\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Science and Research Archive","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.30574/ijsra.2024.12.2.1276","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

摘要

人工智能工具(如 ChatGPT)在学习者中的广泛使用日益普遍,对学术诚信构成了威胁。本研究旨在评估人工智能检测工具在区分人类撰写的作品和人工智能生成的作品方面的能力和效率。通过 ChatGPT 生成了三段关于 "AutoCAD 与建筑 "的作品,并对三段人类撰写的作品进行了评估。使用 GPTZero、Copyleaks 和 Writer AI 等人工智能检测工具对这些段落进行评估。使用 "人类/人类文本/人类生成文本 "和 "检测到的人工智能/人工智能内容 "等参数来评估三种人工智能检测工具在评估输出方面的性能。结果表明,GPT Zero 和 Copyleaks 在确定人类撰写的作品和人工智能生成的作品方面具有更高的可靠性,而 Writer AI 在所有测试输出中显示出显著的 "人类生成内容 "内容分类,在确定人类撰写的作品和人工智能生成的作品方面显示出较低的灵敏度。研究结果表明,在使用人工智能作为人工智能检测工具的同时,还应对结果进行彻底验证和交叉引用。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Comparing ai detectors: evaluating performance and efficiency
The widespread utilization of AI tools such as ChatGPT has become increasingly prevalent among learners, posing a threat to academic integrity. This study seeks to evaluate capability and efficiency of AI detection tools in distinguishing between human-authored and AI-generated works. Three-paragraph works on “AutoCAD and Architecture” were generated through ChatGPT, and three human-written works were subjected to evaluation. AI detection tools such as GPTZero, Copyleaks and Writer AI were used to evaluate these paragraphs. Parameters such as “Human/Human Text/Human Generated Text” and “AI/AI Content Detected” were used to evaluate the performance of the three AI detection tools in evaluating outputs. Findings indicate that GPT Zero and Copyleaks have higher reliability in determining human-authored work and AI generated work while Writer AI showed a notable content classification of “Human Generated Content” on all tested outputs showing less sensitivity on determining human-authored work and AI generated work. Findings indicate that the use of Artificial Intelligence as an AI detection tool should be accompanied with thorough validation and cross-referencing of results.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信