人工智能能取代人类吗?在商业管理教育场景中比较人工智能工具的能力和人类的表现

IF 3 3区 教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH
Dinuka B. Herath, Egena Ode, Gayanga B. Herath
{"title":"人工智能能取代人类吗?在商业管理教育场景中比较人工智能工具的能力和人类的表现","authors":"Dinuka B. Herath,&nbsp;Egena Ode,&nbsp;Gayanga B. Herath","doi":"10.1002/berj.4111","DOIUrl":null,"url":null,"abstract":"<p>This study provides a comparative assessment of the capabilities of leading artificial intelligence (AI) tools and human participants in a business management education context. Specifically, we (a) assess how well current language models perform in providing answers to standardised essay-type assessments in a business and management education context, (b) examine the efficacy of emergent tools in detecting AI-generated texts and (c) evaluate online AI rewriting and paraphrasing tools and their efficacy in evading detection. Using an exploratory qualitative design, this study generated and evaluated 15 standard essays using ChatGPT (<i>n</i> = 5), Bard (<i>n</i> = 5) and human (<i>n</i> = 5). A comparison is provided between the average performance of AI-derived essays and that of ChatGPT-generated essays across all five essays. The results suggest that AI-generated content can achieve reasonably high marks in management and business assessments. According to the findings of the study, AI's performance is highly influenced by the types of prompts used, the user's experience and the degree to which the user can discern between relevant and irrelevant content. According to the findings, Turnitin's AI detection tool is highly effective at detecting content that has been created by AI, but the effectiveness is reduced by rewriters. The Turnitin AI detection tool, however, is significantly more effective at identifying content generated by Bard compared with content generated by ChatGPT. According to the results, ChatGPT produced better results when the user provided a clear context, outlined the topic and expectations, divided the assessment tasks into sections and fed the prompts in a conversational manner to train the model. By utilising AI chatbots effectively, traditional teaching and assessment methods can be supplemented with targeted and engaging learning experiences.</p>","PeriodicalId":51410,"journal":{"name":"British Educational Research Journal","volume":"51 3","pages":"1073-1096"},"PeriodicalIF":3.0000,"publicationDate":"2025-01-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/berj.4111","citationCount":"0","resultStr":"{\"title\":\"Can AI replace humans? Comparing the capabilities of AI tools and human performance in a business management education scenario\",\"authors\":\"Dinuka B. Herath,&nbsp;Egena Ode,&nbsp;Gayanga B. Herath\",\"doi\":\"10.1002/berj.4111\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p>This study provides a comparative assessment of the capabilities of leading artificial intelligence (AI) tools and human participants in a business management education context. Specifically, we (a) assess how well current language models perform in providing answers to standardised essay-type assessments in a business and management education context, (b) examine the efficacy of emergent tools in detecting AI-generated texts and (c) evaluate online AI rewriting and paraphrasing tools and their efficacy in evading detection. Using an exploratory qualitative design, this study generated and evaluated 15 standard essays using ChatGPT (<i>n</i> = 5), Bard (<i>n</i> = 5) and human (<i>n</i> = 5). A comparison is provided between the average performance of AI-derived essays and that of ChatGPT-generated essays across all five essays. The results suggest that AI-generated content can achieve reasonably high marks in management and business assessments. According to the findings of the study, AI's performance is highly influenced by the types of prompts used, the user's experience and the degree to which the user can discern between relevant and irrelevant content. According to the findings, Turnitin's AI detection tool is highly effective at detecting content that has been created by AI, but the effectiveness is reduced by rewriters. The Turnitin AI detection tool, however, is significantly more effective at identifying content generated by Bard compared with content generated by ChatGPT. According to the results, ChatGPT produced better results when the user provided a clear context, outlined the topic and expectations, divided the assessment tasks into sections and fed the prompts in a conversational manner to train the model. By utilising AI chatbots effectively, traditional teaching and assessment methods can be supplemented with targeted and engaging learning experiences.</p>\",\"PeriodicalId\":51410,\"journal\":{\"name\":\"British Educational Research Journal\",\"volume\":\"51 3\",\"pages\":\"1073-1096\"},\"PeriodicalIF\":3.0000,\"publicationDate\":\"2025-01-02\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://onlinelibrary.wiley.com/doi/epdf/10.1002/berj.4111\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"British Educational Research Journal\",\"FirstCategoryId\":\"95\",\"ListUrlMain\":\"https://onlinelibrary.wiley.com/doi/10.1002/berj.4111\",\"RegionNum\":3,\"RegionCategory\":\"教育学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"EDUCATION & EDUCATIONAL RESEARCH\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"British Educational Research Journal","FirstCategoryId":"95","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/berj.4111","RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}
引用次数: 0

摘要

本研究对领先的人工智能(AI)工具和人类参与者在商业管理教育背景下的能力进行了比较评估。具体而言,我们(a)评估当前语言模型在商业和管理教育背景下为标准化论文类型评估提供答案方面的表现,(b)检查紧急工具在检测人工智能生成文本方面的功效,以及(c)评估在线人工智能重写和释义工具及其逃避检测的功效。本研究采用探索性定性设计,使用ChatGPT (n = 5)、Bard (n = 5)和human (n = 5)生成并评估了15篇标准论文。在所有五篇论文中,对人工智能衍生论文和chatgpt生成论文的平均表现进行了比较。结果表明,人工智能生成的内容可以在管理和业务评估中获得相当高的分数。根据这项研究的结果,人工智能的表现在很大程度上受到所使用的提示类型、用户的体验以及用户区分相关和不相关内容的程度的影响。根据调查结果,Turnitin的人工智能检测工具在检测人工智能创建的内容方面非常有效,但重写者的有效性降低了。然而,与ChatGPT生成的内容相比,Turnitin AI检测工具在识别Bard生成的内容方面明显更有效。根据结果,当用户提供清晰的上下文,概述主题和期望,将评估任务划分为部分并以会话方式提供提示以训练模型时,ChatGPT产生了更好的结果。通过有效地利用人工智能聊天机器人,传统的教学和评估方法可以辅以有针对性和引人入胜的学习体验。
本文章由计算机程序翻译,如有差异,请以英文原文为准。

Can AI replace humans? Comparing the capabilities of AI tools and human performance in a business management education scenario

Can AI replace humans? Comparing the capabilities of AI tools and human performance in a business management education scenario

This study provides a comparative assessment of the capabilities of leading artificial intelligence (AI) tools and human participants in a business management education context. Specifically, we (a) assess how well current language models perform in providing answers to standardised essay-type assessments in a business and management education context, (b) examine the efficacy of emergent tools in detecting AI-generated texts and (c) evaluate online AI rewriting and paraphrasing tools and their efficacy in evading detection. Using an exploratory qualitative design, this study generated and evaluated 15 standard essays using ChatGPT (n = 5), Bard (n = 5) and human (n = 5). A comparison is provided between the average performance of AI-derived essays and that of ChatGPT-generated essays across all five essays. The results suggest that AI-generated content can achieve reasonably high marks in management and business assessments. According to the findings of the study, AI's performance is highly influenced by the types of prompts used, the user's experience and the degree to which the user can discern between relevant and irrelevant content. According to the findings, Turnitin's AI detection tool is highly effective at detecting content that has been created by AI, but the effectiveness is reduced by rewriters. The Turnitin AI detection tool, however, is significantly more effective at identifying content generated by Bard compared with content generated by ChatGPT. According to the results, ChatGPT produced better results when the user provided a clear context, outlined the topic and expectations, divided the assessment tasks into sections and fed the prompts in a conversational manner to train the model. By utilising AI chatbots effectively, traditional teaching and assessment methods can be supplemented with targeted and engaging learning experiences.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
British Educational Research Journal
British Educational Research Journal EDUCATION & EDUCATIONAL RESEARCH-
CiteScore
4.70
自引率
8.70%
发文量
71
期刊介绍: The British Educational Research Journal is an international peer reviewed medium for the publication of articles of interest to researchers in education and has rapidly become a major focal point for the publication of educational research from throughout the world. For further information on the association please visit the British Educational Research Association web site. The journal is interdisciplinary in approach, and includes reports of case studies, experiments and surveys, discussions of conceptual and methodological issues and of underlying assumptions in educational research, accounts of research in progress, and book reviews.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信