{"title":"人工智能能取代人类吗?在商业管理教育场景中比较人工智能工具的能力和人类的表现","authors":"Dinuka B. Herath, Egena Ode, Gayanga B. Herath","doi":"10.1002/berj.4111","DOIUrl":null,"url":null,"abstract":"<p>This study provides a comparative assessment of the capabilities of leading artificial intelligence (AI) tools and human participants in a business management education context. Specifically, we (a) assess how well current language models perform in providing answers to standardised essay-type assessments in a business and management education context, (b) examine the efficacy of emergent tools in detecting AI-generated texts and (c) evaluate online AI rewriting and paraphrasing tools and their efficacy in evading detection. Using an exploratory qualitative design, this study generated and evaluated 15 standard essays using ChatGPT (<i>n</i> = 5), Bard (<i>n</i> = 5) and human (<i>n</i> = 5). A comparison is provided between the average performance of AI-derived essays and that of ChatGPT-generated essays across all five essays. The results suggest that AI-generated content can achieve reasonably high marks in management and business assessments. According to the findings of the study, AI's performance is highly influenced by the types of prompts used, the user's experience and the degree to which the user can discern between relevant and irrelevant content. According to the findings, Turnitin's AI detection tool is highly effective at detecting content that has been created by AI, but the effectiveness is reduced by rewriters. The Turnitin AI detection tool, however, is significantly more effective at identifying content generated by Bard compared with content generated by ChatGPT. According to the results, ChatGPT produced better results when the user provided a clear context, outlined the topic and expectations, divided the assessment tasks into sections and fed the prompts in a conversational manner to train the model. By utilising AI chatbots effectively, traditional teaching and assessment methods can be supplemented with targeted and engaging learning experiences.</p>","PeriodicalId":51410,"journal":{"name":"British Educational Research Journal","volume":"51 3","pages":"1073-1096"},"PeriodicalIF":3.0000,"publicationDate":"2025-01-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/berj.4111","citationCount":"0","resultStr":"{\"title\":\"Can AI replace humans? Comparing the capabilities of AI tools and human performance in a business management education scenario\",\"authors\":\"Dinuka B. Herath, Egena Ode, Gayanga B. Herath\",\"doi\":\"10.1002/berj.4111\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p>This study provides a comparative assessment of the capabilities of leading artificial intelligence (AI) tools and human participants in a business management education context. Specifically, we (a) assess how well current language models perform in providing answers to standardised essay-type assessments in a business and management education context, (b) examine the efficacy of emergent tools in detecting AI-generated texts and (c) evaluate online AI rewriting and paraphrasing tools and their efficacy in evading detection. Using an exploratory qualitative design, this study generated and evaluated 15 standard essays using ChatGPT (<i>n</i> = 5), Bard (<i>n</i> = 5) and human (<i>n</i> = 5). A comparison is provided between the average performance of AI-derived essays and that of ChatGPT-generated essays across all five essays. The results suggest that AI-generated content can achieve reasonably high marks in management and business assessments. According to the findings of the study, AI's performance is highly influenced by the types of prompts used, the user's experience and the degree to which the user can discern between relevant and irrelevant content. According to the findings, Turnitin's AI detection tool is highly effective at detecting content that has been created by AI, but the effectiveness is reduced by rewriters. The Turnitin AI detection tool, however, is significantly more effective at identifying content generated by Bard compared with content generated by ChatGPT. According to the results, ChatGPT produced better results when the user provided a clear context, outlined the topic and expectations, divided the assessment tasks into sections and fed the prompts in a conversational manner to train the model. By utilising AI chatbots effectively, traditional teaching and assessment methods can be supplemented with targeted and engaging learning experiences.</p>\",\"PeriodicalId\":51410,\"journal\":{\"name\":\"British Educational Research Journal\",\"volume\":\"51 3\",\"pages\":\"1073-1096\"},\"PeriodicalIF\":3.0000,\"publicationDate\":\"2025-01-02\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://onlinelibrary.wiley.com/doi/epdf/10.1002/berj.4111\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"British Educational Research Journal\",\"FirstCategoryId\":\"95\",\"ListUrlMain\":\"https://onlinelibrary.wiley.com/doi/10.1002/berj.4111\",\"RegionNum\":3,\"RegionCategory\":\"教育学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"EDUCATION & EDUCATIONAL RESEARCH\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"British Educational Research Journal","FirstCategoryId":"95","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/berj.4111","RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}
Can AI replace humans? Comparing the capabilities of AI tools and human performance in a business management education scenario
This study provides a comparative assessment of the capabilities of leading artificial intelligence (AI) tools and human participants in a business management education context. Specifically, we (a) assess how well current language models perform in providing answers to standardised essay-type assessments in a business and management education context, (b) examine the efficacy of emergent tools in detecting AI-generated texts and (c) evaluate online AI rewriting and paraphrasing tools and their efficacy in evading detection. Using an exploratory qualitative design, this study generated and evaluated 15 standard essays using ChatGPT (n = 5), Bard (n = 5) and human (n = 5). A comparison is provided between the average performance of AI-derived essays and that of ChatGPT-generated essays across all five essays. The results suggest that AI-generated content can achieve reasonably high marks in management and business assessments. According to the findings of the study, AI's performance is highly influenced by the types of prompts used, the user's experience and the degree to which the user can discern between relevant and irrelevant content. According to the findings, Turnitin's AI detection tool is highly effective at detecting content that has been created by AI, but the effectiveness is reduced by rewriters. The Turnitin AI detection tool, however, is significantly more effective at identifying content generated by Bard compared with content generated by ChatGPT. According to the results, ChatGPT produced better results when the user provided a clear context, outlined the topic and expectations, divided the assessment tasks into sections and fed the prompts in a conversational manner to train the model. By utilising AI chatbots effectively, traditional teaching and assessment methods can be supplemented with targeted and engaging learning experiences.
期刊介绍:
The British Educational Research Journal is an international peer reviewed medium for the publication of articles of interest to researchers in education and has rapidly become a major focal point for the publication of educational research from throughout the world. For further information on the association please visit the British Educational Research Association web site. The journal is interdisciplinary in approach, and includes reports of case studies, experiments and surveys, discussions of conceptual and methodological issues and of underlying assumptions in educational research, accounts of research in progress, and book reviews.