Exploring ChatGPT's Ability to Classify the Structure of Literature Reviews in Engineering Research Articles

IF 2.9 3区教育学 Q2 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS

IEEE Transactions on Learning Technologies Pub Date : 2024-06-04 DOI:10.1109/TLT.2024.3409514

Maha Issa;Marwa Faraj;Niveen AbiGhannam

{"title":"Exploring ChatGPT's Ability to Classify the Structure of Literature Reviews in Engineering Research Articles","authors":"Maha Issa;Marwa Faraj;Niveen AbiGhannam","doi":"10.1109/TLT.2024.3409514","DOIUrl":null,"url":null,"abstract":"ChatGPT is a newly emerging artificial intelligence (AI) tool that can generate and assess written text. In this study, we aim to examine the extent to which it can correctly identify the structure of literature review sections in engineering research articles. For this purpose, we conducted a manual content analysis by classifying paragraphs of literature review sections into their corresponding categories that are based on Kwan's model, which is a labeling scheme for structuring literature reviews. We then asked ChatGPT to perform the same categorization and compared both outcomes. Numerical results do not imply a satisfactory performance of ChatGPT; therefore, writers cannot fully depend on it to edit their literature reviews. However, the AI chatbot displays an understanding of the given prompt and is able to respond beyond the classification task by giving supportive and useful explanations for the users. Such findings can be especially helpful for beginners who usually struggle to write comprehensive literature review sections since they highlight how users can benefit from this AI chatbot to revise their drafts at the level of content and organization. With further investigations and advancement, AI chatbots can also be used for teaching proper literature review writing and editing.","PeriodicalId":49191,"journal":{"name":"IEEE Transactions on Learning Technologies","volume":"17 ","pages":"1859-1868"},"PeriodicalIF":2.9000,"publicationDate":"2024-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Learning Technologies","FirstCategoryId":"95","ListUrlMain":"https://ieeexplore.ieee.org/document/10547455/","RegionNum":3,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS","Score":null,"Total":0}

引用次数: 0

Abstract

ChatGPT is a newly emerging artificial intelligence (AI) tool that can generate and assess written text. In this study, we aim to examine the extent to which it can correctly identify the structure of literature review sections in engineering research articles. For this purpose, we conducted a manual content analysis by classifying paragraphs of literature review sections into their corresponding categories that are based on Kwan's model, which is a labeling scheme for structuring literature reviews. We then asked ChatGPT to perform the same categorization and compared both outcomes. Numerical results do not imply a satisfactory performance of ChatGPT; therefore, writers cannot fully depend on it to edit their literature reviews. However, the AI chatbot displays an understanding of the given prompt and is able to respond beyond the classification task by giving supportive and useful explanations for the users. Such findings can be especially helpful for beginners who usually struggle to write comprehensive literature review sections since they highlight how users can benefit from this AI chatbot to revise their drafts at the level of content and organization. With further investigations and advancement, AI chatbots can also be used for teaching proper literature review writing and editing.

查看原文本刊更多论文

探索 ChatGPT 对工程研究文章中的文献综述结构进行分类的能力

ChatGPT 是一种新兴的人工智能（AI）工具，可以生成和评估书面文本。在本研究中，我们旨在考察它能在多大程度上正确识别工程研究文章中文献综述部分的结构。为此，我们进行了人工内容分析，将文献综述部分的段落划分为相应的类别，这些类别是基于 Kwan 的模型，该模型是文献综述结构的标记方案。然后，我们要求 ChatGPT 进行同样的分类，并比较了两种结果。数字结果并不意味着 ChatGPT 的性能令人满意，因此作者不能完全依赖它来编辑文献综述。不过，人工智能聊天机器人对给定的提示有一定的理解，并能在分类任务之外做出回应，为用户提供支持和有用的解释。这些发现对于通常难以撰写全面的文献综述部分的初学者尤其有帮助，因为它们强调了用户可以如何受益于该人工智能聊天机器人，在内容和组织层面上修改他们的草稿。随着研究的深入和发展，人工智能聊天机器人还可用于教授正确的文献综述写作和编辑。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

IEEE Transactions on Learning Technologies COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS-

CiteScore

7.50

自引率

5.40%

发文量

审稿时长

>12 weeks

期刊介绍： The IEEE Transactions on Learning Technologies covers all advances in learning technologies and their applications, including but not limited to the following topics: innovative online learning systems; intelligent tutors; educational games; simulation systems for education and training; collaborative learning tools; learning with mobile devices; wearable devices and interfaces for learning; personalized and adaptive learning systems; tools for formative and summative assessment; tools for learning analytics and educational data mining; ontologies for learning systems; standards and web services that support learning; authoring tools for learning materials; computer support for peer tutoring; learning via computer-mediated inquiry, field, and lab work; social learning techniques; social networks and infrastructures for learning and knowledge sharing; and creation and management of learning objects.