通过多模态学习分析解读求助过程:ChatGPT 与人类专家的比较研究

IF 8.9 1区 教育学 Q1 COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS
Angxuan Chen , Mengtong Xiang , Junyi Zhou , Jiyou Jia , Junjie Shang , Xinyu Li , Dragan Gašević , Yizhou Fan
{"title":"通过多模态学习分析解读求助过程:ChatGPT 与人类专家的比较研究","authors":"Angxuan Chen ,&nbsp;Mengtong Xiang ,&nbsp;Junyi Zhou ,&nbsp;Jiyou Jia ,&nbsp;Junjie Shang ,&nbsp;Xinyu Li ,&nbsp;Dragan Gašević ,&nbsp;Yizhou Fan","doi":"10.1016/j.compedu.2024.105198","DOIUrl":null,"url":null,"abstract":"<div><div>Help-seeking is an active learning strategy tied to self-regulated learning (SRL), where learners seek assistance when facing challenges. They may seek help from teachers, peers, intelligent tu-tor systems, and more recently, generative artificial intelligence (AI). However, there is limited empirical research on how learners’ help-seeking process differs between generative AI and hu-man experts. To address this, we conducted a lab experiment with 38 university students tasked with essay writing and revising. The students were randomly divided into two groups: one seeking help from ChatGPT (AI Group) and the other from an experienced teacher (HE Group). To examine their help-seeking processes, we used a combination of statistical testing and process mining methods, analyzing multimodal data (e.g., trace data, eye-tracking data, and conversa-tional data). Our results indicated that the AI Group exhibited a nonlinear help-seeking process, such as skipping evaluation, differing significantly from the linear model observed in the HE Group which also aligned with classic help-seeking theory. Detailed analysis revealed that the AI Group asked more operational questions, showing pragmatic help-seeking activities, whereas the HE Group was more proactive in evaluating and processing received feedback. We discussed factors such as social pressure, metacognitive off-loading, and over-reliance on AI in these different help-seeking scenarios. More importantly, this study offers innovative insights and evidence, based on multimodal data, to better understand and scaffold learners learning with generative AI.</div></div>","PeriodicalId":10568,"journal":{"name":"Computers & Education","volume":"226 ","pages":"Article 105198"},"PeriodicalIF":8.9000,"publicationDate":"2024-11-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Unpacking help-seeking process through multimodal learning analytics: A comparative study of ChatGPT vs Human expert\",\"authors\":\"Angxuan Chen ,&nbsp;Mengtong Xiang ,&nbsp;Junyi Zhou ,&nbsp;Jiyou Jia ,&nbsp;Junjie Shang ,&nbsp;Xinyu Li ,&nbsp;Dragan Gašević ,&nbsp;Yizhou Fan\",\"doi\":\"10.1016/j.compedu.2024.105198\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>Help-seeking is an active learning strategy tied to self-regulated learning (SRL), where learners seek assistance when facing challenges. They may seek help from teachers, peers, intelligent tu-tor systems, and more recently, generative artificial intelligence (AI). However, there is limited empirical research on how learners’ help-seeking process differs between generative AI and hu-man experts. To address this, we conducted a lab experiment with 38 university students tasked with essay writing and revising. The students were randomly divided into two groups: one seeking help from ChatGPT (AI Group) and the other from an experienced teacher (HE Group). To examine their help-seeking processes, we used a combination of statistical testing and process mining methods, analyzing multimodal data (e.g., trace data, eye-tracking data, and conversa-tional data). Our results indicated that the AI Group exhibited a nonlinear help-seeking process, such as skipping evaluation, differing significantly from the linear model observed in the HE Group which also aligned with classic help-seeking theory. Detailed analysis revealed that the AI Group asked more operational questions, showing pragmatic help-seeking activities, whereas the HE Group was more proactive in evaluating and processing received feedback. We discussed factors such as social pressure, metacognitive off-loading, and over-reliance on AI in these different help-seeking scenarios. More importantly, this study offers innovative insights and evidence, based on multimodal data, to better understand and scaffold learners learning with generative AI.</div></div>\",\"PeriodicalId\":10568,\"journal\":{\"name\":\"Computers & Education\",\"volume\":\"226 \",\"pages\":\"Article 105198\"},\"PeriodicalIF\":8.9000,\"publicationDate\":\"2024-11-14\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Computers & Education\",\"FirstCategoryId\":\"95\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0360131524002124\",\"RegionNum\":1,\"RegionCategory\":\"教育学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Computers & Education","FirstCategoryId":"95","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0360131524002124","RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS","Score":null,"Total":0}
引用次数: 0

摘要

寻求帮助是一种与自我调节学习(SRL)相关的主动学习策略,学习者在面临挑战时会寻求帮助。他们可以向教师、同伴、智能辅导系统以及最近的生成式人工智能(AI)寻求帮助。然而,关于学习者寻求帮助的过程在生成式人工智能和人类专家之间有何不同的实证研究十分有限。为了解决这个问题,我们对 38 名大学生进行了一次实验室实验,任务是论文写作和修改。这些学生被随机分为两组:一组向 ChatGPT(人工智能组)寻求帮助,另一组向经验丰富的教师(高等教育组)寻求帮助。为了研究他们寻求帮助的过程,我们结合使用了统计测试和过程挖掘方法,分析了多模态数据(如跟踪数据、眼动跟踪数据和会话数据)。我们的结果表明,人工智能组表现出一种非线性的求助过程,例如跳过评估,这与在高等教育组观察到的线性模型有很大不同,后者也符合经典的求助理论。详细分析显示,人工智能组提出了更多操作性问题,表现出务实的求助活动,而高等教育组在评估和处理收到的反馈时更加积极主动。我们讨论了这些不同求助情景中的社会压力、元认知卸载和过度依赖人工智能等因素。更重要的是,这项研究提供了基于多模态数据的创新见解和证据,有助于更好地理解和帮助学习者学习生成式人工智能。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Unpacking help-seeking process through multimodal learning analytics: A comparative study of ChatGPT vs Human expert
Help-seeking is an active learning strategy tied to self-regulated learning (SRL), where learners seek assistance when facing challenges. They may seek help from teachers, peers, intelligent tu-tor systems, and more recently, generative artificial intelligence (AI). However, there is limited empirical research on how learners’ help-seeking process differs between generative AI and hu-man experts. To address this, we conducted a lab experiment with 38 university students tasked with essay writing and revising. The students were randomly divided into two groups: one seeking help from ChatGPT (AI Group) and the other from an experienced teacher (HE Group). To examine their help-seeking processes, we used a combination of statistical testing and process mining methods, analyzing multimodal data (e.g., trace data, eye-tracking data, and conversa-tional data). Our results indicated that the AI Group exhibited a nonlinear help-seeking process, such as skipping evaluation, differing significantly from the linear model observed in the HE Group which also aligned with classic help-seeking theory. Detailed analysis revealed that the AI Group asked more operational questions, showing pragmatic help-seeking activities, whereas the HE Group was more proactive in evaluating and processing received feedback. We discussed factors such as social pressure, metacognitive off-loading, and over-reliance on AI in these different help-seeking scenarios. More importantly, this study offers innovative insights and evidence, based on multimodal data, to better understand and scaffold learners learning with generative AI.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Computers & Education
Computers & Education 工程技术-计算机:跨学科应用
CiteScore
27.10
自引率
5.80%
发文量
204
审稿时长
42 days
期刊介绍: Computers & Education seeks to advance understanding of how digital technology can improve education by publishing high-quality research that expands both theory and practice. The journal welcomes research papers exploring the pedagogical applications of digital technology, with a focus broad enough to appeal to the wider education community.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信