Can theory-driven learning analytics dashboard enhance human-AI collaboration in writing learning? Insights from an empirical experiment

IF 6.8 1区 教育学 Q1 EDUCATION & EDUCATIONAL RESEARCH
Angxuan Chen , Jingjing Lian , Xinran Kuang , Jiyou Jia
{"title":"Can theory-driven learning analytics dashboard enhance human-AI collaboration in writing learning? Insights from an empirical experiment","authors":"Angxuan Chen ,&nbsp;Jingjing Lian ,&nbsp;Xinran Kuang ,&nbsp;Jiyou Jia","doi":"10.1016/j.iheduc.2025.101054","DOIUrl":null,"url":null,"abstract":"<div><div>The integration of Generative AI (GenAI) into education has raised concerns about over-reliance and superficial learning, particularly in writing tasks in higher education. This study explores whether a theory-driven learning analytics dashboard (LAD) can enhance human-AI collaboration in the academic writing task by improving writing knowledge gains, fostering self-regulated learning (SRL) skills and shaping different human-AI dialogue characteristics. Grounded in Zimmerman's SRL framework, the LAD provided real-time feedback on learners' goal-setting, writing processes and reflection, while monitoring the quality of learner-AI interactions. A quasi-experiment was conducted involving 52 postgraduate students in a human-AI collaborative writing task. The students were divided into an experimental group (EG) that used the LAD and a control group (CG) that did not. Pre- and post- knowledge tests, questionnaires measuring SRL and cognitive load, and students' dialogue data with GenAI were collected and analyzed. Results showed that the EG achieved significantly higher writing knowledge gains and improved SRL skills, particularly in self-efficacy and cognitive strategies. However, the EG also reported increased test anxiety and cognitive load, possibly due to heightened metacognitive awareness. Epistemic Network Analysis revealed that the EG engaged in more reflective, evaluative interactions with GenAI, while the CG focused on more transactional and information-seeking exchanges. These findings contribute to the growing body of literature on the educational use of GenAI and highlight the importance of designing interventions that complement GenAI tools, ensuring that technology enhances rather than undermines the learning process.</div></div>","PeriodicalId":48186,"journal":{"name":"Internet and Higher Education","volume":"68 ","pages":"Article 101054"},"PeriodicalIF":6.8000,"publicationDate":"2025-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Internet and Higher Education","FirstCategoryId":"95","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1096751625000636","RegionNum":1,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}
引用次数: 0

Abstract

The integration of Generative AI (GenAI) into education has raised concerns about over-reliance and superficial learning, particularly in writing tasks in higher education. This study explores whether a theory-driven learning analytics dashboard (LAD) can enhance human-AI collaboration in the academic writing task by improving writing knowledge gains, fostering self-regulated learning (SRL) skills and shaping different human-AI dialogue characteristics. Grounded in Zimmerman's SRL framework, the LAD provided real-time feedback on learners' goal-setting, writing processes and reflection, while monitoring the quality of learner-AI interactions. A quasi-experiment was conducted involving 52 postgraduate students in a human-AI collaborative writing task. The students were divided into an experimental group (EG) that used the LAD and a control group (CG) that did not. Pre- and post- knowledge tests, questionnaires measuring SRL and cognitive load, and students' dialogue data with GenAI were collected and analyzed. Results showed that the EG achieved significantly higher writing knowledge gains and improved SRL skills, particularly in self-efficacy and cognitive strategies. However, the EG also reported increased test anxiety and cognitive load, possibly due to heightened metacognitive awareness. Epistemic Network Analysis revealed that the EG engaged in more reflective, evaluative interactions with GenAI, while the CG focused on more transactional and information-seeking exchanges. These findings contribute to the growing body of literature on the educational use of GenAI and highlight the importance of designing interventions that complement GenAI tools, ensuring that technology enhances rather than undermines the learning process.
理论驱动的学习分析仪表板能否增强人类与人工智能在写作学习中的协作?来自经验实验的见解
将生成式人工智能(GenAI)整合到教育中,引起了人们对过度依赖和肤浅学习的担忧,特别是在高等教育的写作任务中。本研究探讨了理论驱动的学习分析仪表板(LAD)是否可以通过提高写作知识获取、培养自我调节学习(SRL)技能和塑造不同的人类与人工智能对话特征,来增强学术写作任务中的人类与人工智能协作。基于Zimmerman的SRL框架,LAD对学习者的目标设定、写作过程和反思提供实时反馈,同时监控学习者与人工智能互动的质量。我们对52名研究生进行了一项准实验,让他们参与人类与人工智能的协作写作任务。学生被分为实验组(EG)使用LAD和对照组(CG)不使用。收集和分析了知识前后测试、测量SRL和认知负荷的问卷以及学生与GenAI的对话数据。结果表明,EG显著提高了学生的写作知识增益,提高了学生的SRL技能,尤其是在自我效能感和认知策略方面。然而,EG也报告了考试焦虑和认知负荷的增加,可能是由于元认知意识的增强。认知网络分析显示,EG与GenAI进行更多的反思性、评价性互动,而CG则侧重于更多的交易性和信息寻求性交流。这些发现促进了越来越多关于GenAI在教育中的应用的文献,并强调了设计干预措施以补充GenAI工具的重要性,确保技术增强而不是破坏学习过程。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Internet and Higher Education
Internet and Higher Education EDUCATION & EDUCATIONAL RESEARCH-
CiteScore
19.30
自引率
4.70%
发文量
30
审稿时长
40 days
期刊介绍: The Internet and Higher Education is a quarterly peer-reviewed journal focused on contemporary issues and future trends in online learning, teaching, and administration within post-secondary education. It welcomes contributions from diverse academic disciplines worldwide and provides a platform for theory papers, research studies, critical essays, editorials, reviews, case studies, and social commentary.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信