{"title":"比较人工智能探测器:评估性能和效率","authors":"Jeremie Busio Legaspi, Roan Joyce Ohoy Licuben, Emmanuel Alegado Legaspi, Joven Aguinaldo Tolentino","doi":"10.30574/ijsra.2024.12.2.1276","DOIUrl":null,"url":null,"abstract":"The widespread utilization of AI tools such as ChatGPT has become increasingly prevalent among learners, posing a threat to academic integrity. This study seeks to evaluate capability and efficiency of AI detection tools in distinguishing between human-authored and AI-generated works. Three-paragraph works on “AutoCAD and Architecture” were generated through ChatGPT, and three human-written works were subjected to evaluation. AI detection tools such as GPTZero, Copyleaks and Writer AI were used to evaluate these paragraphs. Parameters such as “Human/Human Text/Human Generated Text” and “AI/AI Content Detected” were used to evaluate the performance of the three AI detection tools in evaluating outputs. Findings indicate that GPT Zero and Copyleaks have higher reliability in determining human-authored work and AI generated work while Writer AI showed a notable content classification of “Human Generated Content” on all tested outputs showing less sensitivity on determining human-authored work and AI generated work. Findings indicate that the use of Artificial Intelligence as an AI detection tool should be accompanied with thorough validation and cross-referencing of results.","PeriodicalId":14366,"journal":{"name":"International Journal of Science and Research Archive","volume":"1 12","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-07-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Comparing ai detectors: evaluating performance and efficiency\",\"authors\":\"Jeremie Busio Legaspi, Roan Joyce Ohoy Licuben, Emmanuel Alegado Legaspi, Joven Aguinaldo Tolentino\",\"doi\":\"10.30574/ijsra.2024.12.2.1276\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"The widespread utilization of AI tools such as ChatGPT has become increasingly prevalent among learners, posing a threat to academic integrity. This study seeks to evaluate capability and efficiency of AI detection tools in distinguishing between human-authored and AI-generated works. Three-paragraph works on “AutoCAD and Architecture” were generated through ChatGPT, and three human-written works were subjected to evaluation. AI detection tools such as GPTZero, Copyleaks and Writer AI were used to evaluate these paragraphs. Parameters such as “Human/Human Text/Human Generated Text” and “AI/AI Content Detected” were used to evaluate the performance of the three AI detection tools in evaluating outputs. Findings indicate that GPT Zero and Copyleaks have higher reliability in determining human-authored work and AI generated work while Writer AI showed a notable content classification of “Human Generated Content” on all tested outputs showing less sensitivity on determining human-authored work and AI generated work. Findings indicate that the use of Artificial Intelligence as an AI detection tool should be accompanied with thorough validation and cross-referencing of results.\",\"PeriodicalId\":14366,\"journal\":{\"name\":\"International Journal of Science and Research Archive\",\"volume\":\"1 12\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-07-30\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"International Journal of Science and Research Archive\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.30574/ijsra.2024.12.2.1276\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Science and Research Archive","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.30574/ijsra.2024.12.2.1276","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0
摘要
人工智能工具(如 ChatGPT)在学习者中的广泛使用日益普遍,对学术诚信构成了威胁。本研究旨在评估人工智能检测工具在区分人类撰写的作品和人工智能生成的作品方面的能力和效率。通过 ChatGPT 生成了三段关于 "AutoCAD 与建筑 "的作品,并对三段人类撰写的作品进行了评估。使用 GPTZero、Copyleaks 和 Writer AI 等人工智能检测工具对这些段落进行评估。使用 "人类/人类文本/人类生成文本 "和 "检测到的人工智能/人工智能内容 "等参数来评估三种人工智能检测工具在评估输出方面的性能。结果表明,GPT Zero 和 Copyleaks 在确定人类撰写的作品和人工智能生成的作品方面具有更高的可靠性,而 Writer AI 在所有测试输出中显示出显著的 "人类生成内容 "内容分类,在确定人类撰写的作品和人工智能生成的作品方面显示出较低的灵敏度。研究结果表明,在使用人工智能作为人工智能检测工具的同时,还应对结果进行彻底验证和交叉引用。
Comparing ai detectors: evaluating performance and efficiency
The widespread utilization of AI tools such as ChatGPT has become increasingly prevalent among learners, posing a threat to academic integrity. This study seeks to evaluate capability and efficiency of AI detection tools in distinguishing between human-authored and AI-generated works. Three-paragraph works on “AutoCAD and Architecture” were generated through ChatGPT, and three human-written works were subjected to evaluation. AI detection tools such as GPTZero, Copyleaks and Writer AI were used to evaluate these paragraphs. Parameters such as “Human/Human Text/Human Generated Text” and “AI/AI Content Detected” were used to evaluate the performance of the three AI detection tools in evaluating outputs. Findings indicate that GPT Zero and Copyleaks have higher reliability in determining human-authored work and AI generated work while Writer AI showed a notable content classification of “Human Generated Content” on all tested outputs showing less sensitivity on determining human-authored work and AI generated work. Findings indicate that the use of Artificial Intelligence as an AI detection tool should be accompanied with thorough validation and cross-referencing of results.